BrainPort for the Blind

Arguably, a plug in the eyes is ideal technology to help blind people navigate. Imagine a video camera fixed directly into the brain, giving sight to the sightless, changing their world profoundly.

That technology—or something very similar—is available now. Only it does not plug into the eyes or brain. It plugs into the tongue.

The BrainPort, as the product is known, is the work of Paul Bach-y-Rita, a professor of Rehab Medicine and Biomedical Engineering at the University of Wisconsin-Madison. In the 1960s, he created a machine that converted images from a camera to vibrations on plates. Wearing the plates on their backs, blind volunteers would try to decode the images.

In the 90s, Bach-y-Rita revamped the technology. Instead of the back, which has few nerve endings, he switched to the tongue. Not only does the tongue have a high density of nerve endings, but the absence of skin also allows electrodes to more directly stimulate receptors, and its saliva makes for better conductivity.

Bach-y-Rita died in 2006, but the company he founded, Wicab, continued to work toward better usability of his device. Early prototypes involved a 12x12 grid of electrodes that connected to a video camera worn on the forehead. The data from the camera was turned into, essentially, a low-resolution image of 144 pixels. Where there was light, electrodes buzzed the tongue.

Since then, the device has come a long way. The number of electrodes on the tongue—and the resolution of the translated images is now 20x20. “DARPA gave us funding to explore what the limits were, in terms of spatial and temporal resolution,” said Richard Hogle, Wicab’s director of product development.

Few people, they found, could tell the difference between 20x20 and 25x25 grids. With resolutions as high as 40x40, but the difficulty of manufacturing that many electrodes in an area small enough to fit on the tongue became prohibitive. Also, the BrainPort accentuates edges, assisting users interpreting the data they experience coming into their tongues. Bach-y-Rita’s original prototype placed cables from each electrode in the mouth to the controller, 144 total.

Recommended for You: Wearables Help the Blind Walk

“As you increase the number of electrodes, you have to increase the number of cables as well, or you have to come up with a different scheme,” said Hogle. “The different scheme is to put more electronics in the mouth, and have a single cable that sends patterns up to the electronics in the mouth.”

All those components now fit in a postage stamp-sized square, about a quarter of an inch thick. “Kind of like a lollipop,” said Hogle. A single cable runs from the lollipop to the headset with the camera.

Other changes the BrainPort has seen reflect the needs of the target users. Because of the low resolution of the image on their tongues, users can zoom. Moving the head around and zooming in and out provides a sense of dimension (i.e., things that are closer will change faster) and how things are arranged in space

Wicab also found that users did not want a controller they had to hold in a hand. With a cane, a phone, and—possibly—a dog to deal with, they did not want “another thing on the utility belt to be aware of,” said Hogle. So he and his colleagues worked to put everything—controllers, electronics, and battery—in the headset.

Training the User

Of course, blind users don’t just slip on a BrainPort and suddenly recognize their surroundings. “We have a camera that transfers the 3D world into a 2D world, turning it into stimulation patterns on the tongue,” said Hogle. “It takes training to interpret those patterns into useful information.”T

The trainers are necessarily sighted. Therefore, the BrainPort—which has no LEDs or screens for blind users—needed a visual display for trainers to help users interpret what they were sensing on their tongues. The solution was a Wi-Fi interface, enabling trainers to see in real time what trainees were experiencing.

You May Also Like: Wearable Robot Helps Children Walk

Future models of the BrainPort will help users understand their surroundings even more. “For instance let the user know about signs in the vicinity,” said Hogle. “Imagine walking down the street. As they are gazing—pointing the camera—if there is a sign of interest, the system can let them know.”

The image from the headset could be streamed to a phone, which could process the reading of a sign or similar task. Similarly, the BrainPort could someday use the technology helping self-driving cars navigate, to actually help blind people navigate.

Though the BrainPort doesn’t offer full sight to blind people, it has given many users experiences they would not otherwise have had. Many have been able to walk down sidewalks in a straight line, unassisted; some have tried activities like playing darts, bowling, and pool. One congenitally blind individual said the device gave him a sense of what perspective means.

“From an independence perspective,” said Hogle, “that’s a pretty big deal.”

Michael Abrams is a technology writer based in Westfield, New Jersey.