95 Sensory Substitution

Sensory substitution is the process of using a different sense (such as touch) to replace or makeup for the lack of another (commonly sight). Often technology is required to make this process more feasible and provides an alternative to those who lack the ability to experience sensations in the usual way.

There are many tools for visually impaired people on the market that can help them live more independent lives. Early devices included the BrainPort, which looks a little bit like a lollipop and is a device that provides tactile stimuli to the tongue to give the user information about the visual landscape. As its inventor, Paul Bach-y-Rita pointed out, “we see with our brains, not our eyes”. Using our other senses to provide spatial information to the brain can, in some cases, invoke useful visual experiences. Below we list a few more approaches that take advantage of our ever-expanding network of technology, cameras on phones, and an army of dedicated volunteers.

One example is Microsoft Seeing AI (https://www.microsoft.com/en-us/ai/seeing-ai), which is an application of current developments of computer vision. The app can be used in different contexts, such as reading, scene recognition, and social interaction. It can transcribe printed text into audio by using the phone camera. People can also use it to recognize people’s faces, currency, and some common scenes.

Something similar to Microsoft Seeing AI is an app called Be My Eyes that vision impaired people can download and use in places such as the grocery store or bookstore. Using the smartphone camera, users can ask for support from volunteers by showing what they would like described. Be My Eyes has also started using AI where users can take a picture of something and get a description using the app as well. (more information in this YouTube video about Be My Eyes)

Another approach is the service of remote assistants, such as Aira (https://aira.io/). Users who subscribe to the service can connect with a trained assistant on their phones. The agent can see through a user’s phone camera and offer requested assistance for activities such as reading mail/bills or navigation. Many public spaces in the U.S. have adopted this service. For example, people with visual impairment can access this service in the MSP airport and many Target stores without additional charges.

VoiceOver is also a popular accessibility function built in Apple products, such as iPhones, iPads and Macs. It can help users read texts on their devices.

These technologies cannot replace white canes or guide dogs or orientation training for the visually impaired, but they can provide more accurate information for the visually impaired about their surrounding environment and the elements they interact with on a daily basis.

If you want to know more about sensory substitution, watch this video or take a look at this website.