Program Lets Disabled People Compose Music With Their Eyes

Thanks to EyeHarp, people with severe motor disabilities can now perform and compose music by gazing at a screen

Photo Illustration: R. A. Di Ieso
Jun 24, 2016 at 7:00 AM ET

Historians agree that Beethoven was almost entirely deaf when he wrote his famous Moonlight sonata, and entirely unable to hear for much of his late period—just one famous example of why disabilities don’t need to end an artistic career.

But for people with ALS and other severe motor handicaps, the lithe movements of a musician and the precision of a composer can seem unattainable. Enter EyeHarp—a new computer program that allows users to perform and compose music, using only their gaze as a control mechanism.

Scientists reviewed EyeHarp in a new study published in Frontiers In Psychology. They found that the device was technically on par with conventional instruments and capable of producing high quality musical pieces. “The practice required to master the EyeHarp DMI is similar to the average practice required to master a traditional musical instrument of average difficulty,” the authors write. “[EyeHarp allows users] to produce expressive performances both from the performer and audience perspective.”

Music isn’t just a fun hobby. Prior studies haven shown that people who learn to play a musical instrument report overall higher quality of life and enjoy a large number of cognitive and psychological benefits. People with severe disabilities could really benefit from those kinds of advantages—if only they had the fine motor skills to play the piano.

The notion of customizing musical instruments for people with severe disabilities has been around for some time, in the form of Adaptive Digital Musical Interfaces or ADMI. For instance, Skoog, one of the most prominent ADMI platforms, is an inexpensive cube that users with cerebral palsy and similar conditions can press to perform basic musical pieces. But for ALS patients and others who fall under the broad umbrella of locked-in syndrome—an inability to move anything but the eyes—even operating a Skoog device is an insurmountable challenge. For these aspiring musicians, researchers have been developing eye-controlled musical devices since at least 2014.

But EyeHarp is by far the most exciting prospect. EyeHarp was built to mimic other high-end musical instruments—not to simply replicate or simplify the process of composing and performing, but to function as efficiently as a piano or a guitar. And indeed, EyeHarp allows users to control chords, arpeggios, melody, and loudness using only their gaze.

The question, of course, is how well it really works. Is the EyeHarp a serious instrument with an unconventional input, or is it just a fancier version of Skoog? To find out, the developers of EyeHarp recruited eight musicians (who did not have any disabilities) to learn to play their unconventional instrument, along with 31 audience members (also without any disabilities) who agreed to rate each performance.

The results broadly suggest that EyeHarp is, as expected, different—but functionally as capable, challenging and gratifying as any other musical instrument. Performers reported that, “the practice required to play the EyeHarp is comparable to the practice required to play a traditional musical instrument of average difficulty” and that they felt the had the same control over the musical input as they would over any other instrument. Meanwhile, almost all of the performers noted that playing music with the eyes is more tiring than playing with the hands, and that keeping tempo with the eyes is a challenge.

“We could conclude that performing music with the eyes is more difficult that performing with traditional means,” the authors write. “Nevertheless, learning the EyeHarp gaze-controlled musical instrument wouldn’t be harder than learning a traditional musical instrument.”

Perhaps most importantly, the music produced by these performers was high-quality. All of the audience members said that they thoroughly enjoyed the performances, with the average participant rating the concerts 4.3 out of 5.

The authors says that the obvious next step will be testing EyeHarp on musicians who have lost motor control, and that they have already begun actively recruiting participants for that trial. In the mean time, the researchers are optimistic that eye-tracking technology will continue to advance, ultimately helping people with disabilities to escape the confines of their bodies, through music, communication and self-expression.

“The cost of eye-tracking technology decreases every year,” the authors write. “Eye-tracking is slowly being incorporated in common place laptops, tablets and mobile phones. Such devices would allow many users, including users with motor disabilities, to have access to gaze-controlled applications, including the EyeHarp.”