Adding Textures to Touchscreens

Glass panes, the kind that form the fronts of most smart phones and tablets, are noted for being smooth. That smoothness can be a problem, though, for users trying to operate virtual buttons or knobs projected on the device’s display. Now, researchers at Northwestern University believe they understand how to provide tactile feedback on a sheet of glass, so that a computer screen could not only show what something looks like, but provide a sense of how it feels.

Haptic displays are not new. About 10 years ago, researchers at Northwestern’s mechanical engineering department developed a device that could display virtual shapes and textures—like buttons, bumps, depressions, or stickiness—over a touchscreen. The device, called TPaD, applied ultrasonic waves to a thin glass plate placed over the screen and decreased the friction between the fingertip and the plate, making the glass feel slippery.

But to make the device generally useful, the researchers had to understand why it worked.

There were two competing hypotheses. One stated that the vibrating plate caused the thin film of air under the fingertip to compress and create pressure that levitates the skin of the finger off of the screen. The second proposed that ultrasonic vibrations simply cause the skin to bounce off the glass surface.

Northwestern’s Michael Wiertlewski created a test device to settle the matter. Wiertlewski built a fingerprint imager, similar to the kind used for security, and attached it to a TPaD. Using a special strobe light and high-speed camera, he measured the friction felt by a finger as it slid across the glass plate.

His results, which were published in the Proceedings of the National Academy of Sciences, supported both theories, in part. The measurements showed the reduction of friction, or the slipperiness, is caused by the skin bouncing—not on the glass plate, but on a layer of air trapped between the plate and the surface of the finger.

“This paper proves the fingertip is actually bouncing on air,” Wiertlewski said.

Wiertlewski’s team will use the research to develop new algorithms to create additional and more accurate textures, and to improve the energy consumption of new types of haptic devices.

The team also hopes to use the vibrations to push or guide fingertips across the screen. That could be used to align the fingers of the vision-impaired over a keyboard, said mechanical engineering professor Ed Colgate, a haptics pioneer who supervised the team.

Last year, two of Colgate’s grad students created and began shipping the TPad Phone, a customized smartphone that researchers and designers can use to create their own textures and shapes.

Similar applications building on Wiertlewski’s work could be used by online retailers to display the texture of a new pair of leather shoes or a wool sweater, or by car manufacturers to create virtual buttons and knobs on the large touchscreens that are gradually taking over car dashboards.

Those textures and shapes won’t necessarily feel like an exact copy of what they’re trying to replicate. But that’s okay, Colgate said, because the human brain can fill in the gaps with what it’s already familiar with.

“It’s never going to be perfect,” Colgate said. “But if we can get close, your brain will make its own conclusions.”

Read the latest issue of the Mechanical Engineering Magazine.

For Further Discussion