New App Makes It Easier Than Ever For ALS Patients To ‘Speak’

A Microsoft app aims to fix the most frustrating problems of communication

Feb 21, 2017 at 4:12 PM ET

It’s hard to imagine what could be more frustrating than being unable to communicate your thoughts through speech or writing. But for people with motor neuron diseases like ALS, also commonly called Lou Gehrig’s disease, making sure a simple thought is heard can be a logistical nightmare, even with the aid of new communication technologies that track eyeball movement. Now, researchers have developed a new approach they think can help.

GazeSpeak, a mobile phone app created by researchers at Microsoft, aims improve on traditional and digital devices already on the market that allow a person’s eye movements to be translated into words.

To use the app, those seeking to listen point their smartphone at the person “speaking.” The speaking person uses simple up, down, left, and right eye signals corresponding with the user key sticker stuck to the back of the phone while looking towards the camera. Using computer vision to track eye movements and predictive software that guesses at words from the first few letters and users’ past word choices, GazeSpeak quickly converts flicks of the eye into speech.

The app builds upon the augmentative and alternative communication concepts first developed in the 1950s in the form of transparent, low-tech eye gaze communication boards, then called e-tran boards, which are still used in certain situations. Eye tracking communication devices have come a long way since then, with digital versions allowing for far more expedient conversation and even helping users to directly control their environments.

Still, the researchers say that user experiences with existing technologies are far from perfect. In surveying 22 partners and caregivers of people with ALS, they found that the challenges they experienced using digital devices made for this purpose included system crashes, interference from sunlight, and their lack of portability. Additionally, these devices can be quite expensive, ranging from $5,000 to $10,000, leaving some of those in need to have to purchase or crowdfund what it is they need to communicate with the outside world.

When tested, GazeSpeak was found to have 90 percent accuracy among able-bodied participants, and both speakers and their caregivers learned to use it quickly during informal demo sessions. While GazeSpeak is not yet publicly available, Microsoft say the app (and its source code) will be on the Apple App Store within the coming months.