Thanks to a handful of emerging technologies, virtual touch-screen keyboards are getting closer to the feel of real electromechanical keyboards. Enhancements such as tactile feedback and surfaces that change to mimic physical keys could eventually redefine the virtual keyboard experience for millions of users of devices ranging from smartphones to tablets and touch-screen PCs.
Will these improvements be enough for the virtual keyboard to entirely displace the electromechanical keyboard? Maybe not for folks old enough to have used an IBM Selectric typewriter -- whose keyboard served as the model for early computer keyboards -- but improved virtual keyboards may be just fine with a new generation of users for whom big clunky keyboards are so yesterday.
On smartphones, virtual keyboards have largely replaced the more expensive electromechanical keyboards, with a few notable exceptions such as BlackBerries and QWERTY texting phones.
On the PC front, we've seen concept laptops with two displays for some time now, usually with the idea that the bottom screen can be used as a virtual keyboard when needed. And PC vendor Acer recently announced a dual-screen notebook as well, though such devices are still far from mainstream. On PCs used for intensive content creation, however, the physical keyboard is unlikely to go away entirely, although an integrated virtual keyboard is likely to become a complementary component.
The real battleground may be over tablet computers. As users of devices such as the iPad progress from Web surfing and content consumption to a mix of consuming and creating content, demand for better keyboard performance will increase.
Today most iPad users who buy productivity software also reach for Apple's optional external keyboard, says the sales manager at one Apple Store, and about 40% of those who come into that store for iPad training at the Genius Bar bring in or walk out with external keyboards. Tomorrow, though, the touch screen may just be good enough.
Good vibrations
Next-generation touch-screen devices will embed more haptics, or touch-based feedback, into virtual keyboards. Haptic technology uses targeted vibrations to deliver tactile feedback that can vary in frequency, direction and intensity to simulate a key click or to present different surface textures within discrete areas of the display.
When combined with visual and audio input, those finely tuned vibrations, which may be generated by mechanical actuators or electrostatic charges, can fool your brain into thinking that you've just pressed a physical key.
"A lot of companies are really getting into haptics, [using] source feedback and a sense of touch to try to replicate a keyboard on a display," says Bruce Gant, a mechanical engineer at Product Development Technologies, which integrates touch screens into cell phones and other devices for manufacturers. "If people really get that down and nail that experience, [virtual keyboards] could replace mechanical keyboards on laptops."
Immersion Corp., which has developed a product that uses a mechanical actuator to deliver haptic feedback on a touch screen, says the next generation of haptics will be able to replicate the feel of key travel as well as the keyboard click of a mechanical keyboard. "You don't get the actual travel of your finger, but you can get much more of it back," says Dennis Sheehan, Immersion's vice president of marketing.
But simple tactile feedback isn't enough for touch typists. That's because the feel of a real keyboard goes beyond the limited range of surface textures that haptics can provide. For example, a touch typist continuously realigns his fingers by sensing the edges of each key as he types. The gaps between the keys and the bumps on the J and F home keys let him find the correct position by feel.
"On a touch screen, you lose all of that surface feel," says Mike Levin, a vice president at haptics technology provider Pacinian Corp.
Even with haptics technology, touch typists won't be able to feel a tactile ridge along the edge of virtual keys, as they would on physical keyboards. But Pacinian's pressure-sensitive haptics can provide a different sensation for the home keys to differentiate them from the other keys on the keyboard, Levin says.
Unfortunately, today Pacinian's technology can only do that for one position at a time: You can sense a different pulse for the J or the F position, but not for both at once. Pacinian is working on a solution for that.
Force sensing, surface deforming
A bigger problem is that on today's capacitive touch screens, which detect your fingers' electrical conductivity as you touch the screen, you can't touch the keyboard to orient your fingers without activating the keys. On a physical keyboard, you touch the keys to determine position and then press down on them to type. That's missing on touch screens, which can sense a touch, but not the force with which it is applied.
Pacinian is also working to improve that. "The stuff we're working on adds a force sensor to improve the experience," says Levin. In this way, a user could use touch to find a button and then increase pressure to select it.
If Pacinian can provide feedback for both home key positions and use force-sensing technology to allow users to orient their fingers without activating keys, a virtual keyboard might be good enough for hunt-and-peck typists. But it's still not going to be enough for high-speed touch typists who are accustomed to rolling along at 100 to 150 words per minute. "You don't have those surface features to locate your fingers on," Levin says. So users would still have to look at the keyboard as they adjust their finger positions.
Pursuing a moving target
In trying to offer better tactile response for typists, virtual keyboards are following a moving target. The essential characteristics of the electromechanical keyboard have changed dramatically over the past 20 years, says David Hill, vice president of design at Lenovo.
The keyboard that graced the original IBM PC 5150 in 1981 was designed to mimic the feel of an IBM Selectric typewriter, which in turn was designed to mimic the feel of earlier typewriters, from the QWERTY layout to the long key "travel" that allowed the typebars (thin arms with letters on them) to strike the platen (the roller on which the paper was fed).
Early electric typewriters offered a loud response and strong "force curves," measures of the pressure needed to depress a key and the speed at which it returns to its original state. Keyboards weren't designed for ergonomics and performance so much as to be compatible with what was already familiar to typists of the day.
"Over time, those have been transitioned to keyboards that have less sound, less clickety clackety and less travel," Hill says.
Today's keyboards mount mechanical keys over electric sensors, most often a two-layer membrane with a suspension layer in between. When the user pushes down, a key depresses the layers, making a contact. While the membrane itself has a certain amount of spring to it, the device may also include a spring or, in the case of Lenovo's ThinkPad notebook line, a scissoring mechanism that snaps the keys back in place after the user depresses them. Key travel can be as short as 2mm, about half that of early designs.
Keyboard preferences are very much generational, so the acceptability of virtual keyboards depends on the user's frame of reference. The attributes that were important to a typist familiar with the manual typewriters of 1940s are different from what's important to a secretary who used a Selectric in the 1960s. Today's young users have an entirely different perspective. Most are already familiar with touch-screen keyboards and have never seen or used an electric typewriter, never mind a manual one. And most would find the keyboard that shipped with the original IBM PC to be clunky, says Hill.
So virtual keyboards don't necessarily have to emulate today's electromechanical keyboards in every respect. For example, Hill says, "there's an ergonomic improvement with shorter travel because you can type faster," but only to a point. While fingers striking a solid glass surface offer no travel at all -- not very good for touch typing -- virtual keyboards don't have to emulate the long travel of the previous generation of keyboards to offer acceptable performance.
Immersion is working on an innovation that could help. The company is collaborating with partners to develop "deformable surfaces" that would allow certain areas of the touch screen to rise up, creating key positions you can feel.
"When electric fields are applied to them, they deform -- raise up or sink down. Those surfaces would be on top of the touch-screen display," says Sheehan, so this could help users position their fingers by feel. But don't expect products anytime soon: Sheehan describes the work as part of the company's "long-range vision."
Make that very long-range vision, says Levin. Pacinian has researched different deformable materials, including one called electroactive polymers. Most of the work in developing deformable surfaces, Levin says, is still in university research labs, although several companies, including Microsoft, Apple and Nokia, have filed for patents in this area.
"We have put that aside for the near term, as they still have problems with robustness and reliability, especially across varying temperatures and humidity. It's unclear which of the early concepts will actually work, much less [be] able to be manufactured reliably and cost effectively," Levin says.
Then there's the sticky issue of key travel. On a touch screen, your fingers are pressing on a hard glass or plastic surface. Haptics can simulate key travel to an extent, but the technology today is very limited.
Pacinian's "force sensing" electrostatic surface actuation technology comes into play here as well. The sensor can tell which keys you're pressing and how hard, and apply feedback that may help create a sensation of movement, Levin says.
But Pacinian's technology won't work for an entire virtual keyboard on a touch screen because the pressing sensation as engineered today isn't granular: It gets applied to the entire touch surface rather than to a specific area of it. It can tell the position on which your fingers are pressing, but it can only do force sensing for one position -- one finger -- at a time.
This limitation derives from the fact that most touch-screen displays today use a single sheet of glass as the top layer. Pacinian is researching designs that use a plastic top layer, which allows for multifinger pressure sensitivity. "A flexible surface can have regional sensing and actuation," Levin says. But there's a trade-off: Plastic is less transmissive than glass, so the screen may not be as bright.
Getting to 'good enough'
The virtual keyboard experience, which is not great today, will get much better, the experts say. But will it ever be good enough to replace a traditional keyboard for intensive data entry on a desktop or laptop?
Jennifer Colegrove, an analyst with display market research firm DisplaySearch, thinks it will. "But both hardware and software need to improve to reach that point, and price reductions are needed for adoption," she says. For now, she adds, touch and mechanical keyboards will coexist for use with laptops and other computers.
As for when virutal keyboards might replace mechanical ones, that's likely still years away. "Some notebooks and all-in-one PCs that only use touch-screen keyboards have already been demonstrated, or patents have been applied for them. I expect those to enter the market within 12 months, but high volume is still five or more years away," says Colegrove. She predicts that virtual keyboards will be good enough for touch typists in 10 years.
Other experts aren't so sure that a virtual keyboard can ever replace a good mechanical one -- particularly for touch typists. Ken Bosley, software product manager for Hewlett-Packard's Consumer Desktop Global Business Unit, says HP hasn't seriously considered including haptics in the company's TouchSmart line of capacitive touch-screen PCs.
Both laptops and desktops in the TouchSmart line offer virtual keyboards, and the display on the desktop version tilts to 30 degrees for easier typing. But typing on it is still awkward, and Bosley doesn't think most people use the virtual keyboard since a physical one is readily at hand. He says it's there only because users have come to expect it on touch screens.
He doubts screen-based virtual keyboards will ever replace electromechanical models for desktop PCs and laptops. "The reason I don't like virtual keyboards is that I can't touch type on them," he says.
Andrew Hsu, technology strategist at touch-screen maker Synaptics Inc., also doesn't see virtual keyboards dislodging their physical cousins. "Some of the most difficult problems for haptics are simulations of mechanical devices," he says, and there will continue to be a need for electromechanical keyboards for the foreseeable future.
But, Hsu adds, to try to replicate the experience of a mechanical keyboard on a touch-screen surface is to miss the point. "We could enable more realistic keyboard input if that's what we continue to require five years from now," he says, but why would touch-screen makers want to replicate the usage scenarios and paradigms from the mechanical keyboard era?
Hsu points to gesture-based applications such as Swype, which lets users "type" by sliding a finger across a virtual keyboard. "With a lot of gesture-based interactions, having a mechanical input device like a keyboard becomes almost unnecessary. You can already see a lot of kids adopting it very easily," Hsu says.
In some ways a touch screen is a better interface than a keyboard, says Scott Birnbaum, vice president of LCD business at Samsung, touting the company's use of Swype technology on its Galaxy S smartphone and Galaxy Tab tablet. "Instead of pushing a button to get a letter, you gesture with a finger," he says. "To type hello you move your finger from the H to the E and then circle around the L for two L's" and end on the O, he says. "Now that I've used Swype, I'll never go back to a keyboard."
Watch this video to see how Swype works.
But Bosley, who worked on HP's original touch-screen technology in 2007, is skeptical. "To me, Swype is just using a keyboard in a more efficient way," he says. While users like cool gestures like pinch and swipe, he says consumers' appetites for more sophisticated gesture libraries will be severely limited. There's a reason why we're not still writing in shorthand, he says: "The typewriter was easier and less tiring."
Mike McSherry, CEO of Swype, says gesture languages complement keyboards, but aren't substitutes for them. "It's not realistic to expect the average user to learn more than a couple-dozen gestures," he says, whereas people tend to use a vocabulary of over 60,000 English words.
Gestures, he says, will stick to the more traditional role of device navigation and selection metaphors, such as turning a page or launching an application. "Like voice input, gestures complement keyboard usage activity but will not replace the need for keyboard text entry."
On the other hand, touch screens offer features that mechanical ones don't, such as real-time error correction and word prediction. And "touch-screen keyboards will add new functions, such as overlays to show images or other information and [support for] handwriting, signatures, etc., which the traditional keyboard can't offer," says DisplaySearch's Colegrove.
A generational thing
Total replacement of the physical keyboard is achievable, Immersion's Sheehan insists. But, he asks, "Is that a relevant question? Children who have grown up with touch-screen keypads don't distinguish the difference. They just deal with it."
Hsu sees the same trend. Early on, Synaptics thought that text entry was a big problem on mobile phones that had simple numeric keypads, and proposed a full QWERTY keyboard solution. It went nowhere. Manufacturers, who knew their customers, laughed off the idea.
"An entire generation of handset users were quite comfortable with doing texting on a 12-key keypad," Hsu says. "As we approached developers, they said, 'That's a problem for the older generation.'"
Similarly, Hsu continues, "There's a perception among the older generations that grew up on keyboards that we would miss a keyboard. But as newer generations are raised on devices that have a gesture-based interface they just won't care," he says.
Bosley, for one, thinks younger users do want real keyboards on PCs -- and even on cell phones. "The things my kids asked for after they had a phone for a while was a phone with a fold-out QWERTY keyboard because they text a lot," he says. But some of the most popular smartphones, such as the iPhone and many Android phones, rely solely on virtual keyboards.
Even if virtual keyboards don't displace physical ones in all situations -- and they won't -- touch screens and haptics may change how we interact with a keyboard -- especially on tablets, says Hsu. "The biggest challenge will be for those of us who grew up with the keyboard to abandon interaction scenarios that are near and dear to our hearts, but not more usable."
Robert L. Mitchell is a national correspondent for Computerworld. Follow him on Twitter at Twitter twitter.com/rmitch, or e-mail him at rmitchell@computerworld.com.
Join the CIO Australia group on LinkedIn. The group is open to CIOs, IT Directors, COOs, CTOs and senior IT managers.