Be A Maverick. The University of Texas at Arlington
Helping the Blind See
Electrical engineering doctoral student Ninad Thakoor has embarked on an ambitious project—providing the blind and visually impaired with a wearable, interactive system that will better inform them of their indoor and outdoor surroundings.
For decades, researchers have created a broad, but often impractical, assortment of assistive devices to supplement or replace the cane as a primary sensory feedback mechanism for the blind and visually impaired. Now, thanks to advanced technologies and clever thinking, Thakoor and a team assembled by computer science and engineering Assistant Professor Jean Gao have created a working prototype of a device that improves sensory information for the blind and visually impaired.
Called IntelligentEyes, the system has four major components: eyeglass frame-mounted cameras providing real-time pattern recognition and distance information; a wireless, ultra-wideband transceiver sending video and voice signals to and from a reconfigurable processor; a battery-powered image processor analyzing information from the cameras; and voice recognition and generation software providing ease-of-use communications from and to the user. The prototype contains all but the system’s voice capabilities.
Thakoor, a research assistant in the Computer Science and Engineering Department’s BioComputing and Vision Lab, has worked on the system for several years using some off-the-shelf hardware components and others created specifically for the project. He also helped develop the video analysis and other software powering the system.
“The IntelligentEyes system won’t replace the cane,” he said. “It would be an additional source of important information for a blind or visually impaired user, information on things beyond the length of the cane.”
Wireless signals from the eyeglass-mounted cameras are processed to provide several types of information about what is in front of the user. The signals may indicate a flat surface, a person, an unknown object or a change in surface like a step or curb. They also provide the distance to that object or surface change, its movement and the rate of speed of that movement.
This information is then processed through the voice generation software, creating a vocal response sent to a speaker mounted in the eyeglass frame or an earpiece. The user will be able to tell the processor how frequently information should be updated and presented.
Dr. Gao and Dr. Richard Srebro of the Ophthalmology Department at UT Southwestern Medical Center at Dallas visited the Dallas Lighthouse for the Blind, a non-profit organization providing social services and employment opportunities for the blind and visually impaired.
“I needed to learn directly from these individuals the kinds of information and feedback they wanted from an assistive device,” Gao said. “During our visit, I was continually amazed by the groups of workers assembling different products and mechanisms. It was also interesting to see how they made use of and depended on the senses they had to do their work and move about the facility.”
Thakoor says the system is still in its infancy. The image processing has been refined to target single objects, much as a sighted person’s vision, but disregards peripheral ones. And though the wireless signal transmission is working as planned, the individual units are not yet packaged for consumer use.
“This is such an exciting project,” he said, “one that we think we can achieve, one that will really make a dramatic improvement in the lives of the blind and visually impaired.”