SEATTLE
A mouse that can move accurately in response to sounds could mean the difference between dependence and independence for someone with motor impairments.
A University of Washington researcher with a specialty in speech recognition is putting his skills to use teaching computers to do just that.
Jeff Bilmes, the associate professor of engineering leading a team creating the Vocal Joystick with a grant from the National Science Foundation, said most existing controllers are far from ideal.
Some are controlled by the breath or the tongue, but then users can’t talk while operating their PC, and if the device falls out, someone else has to put it back in.
“The one last bit of independence that such individuals have has been lost,” Bilmes said.
Other solutions, like sensors implanted under the skin or eye-movement trackers, can be invasive or expensive, he noted. Vocal Joystick is neither.
Bilmes said he was inspired by “beatboxers,” people who make a wide array of percussive noises with their mouth.
“The human voice can do much more than what’s typically done for a standard language,” said Bilmes, who presented Vocal Joystick at a National Academy of Engineering meeting this week.
His team assigned vowel sounds to correspond to mouse movements: Up is “a” as in cat, right is “aw” as in “Aw, shucks.” There are sounds for more complex movements “down and to the right” is “oh” as in bold and sliding sounds together sends the mouse on smooth curves around the screen.
Vocal Joystick also gives computer users control over the mouse pointer’s speed. A very quiet “ee” as in beet will move the cursor slowly to the right; a loud “oo” as in boot sends it plunging downward.
Once users are accustomed to the commands, they can do sophisticated on-screen work, such as illustration or digital photo restoration.
When he first started thinking about it, Bilmes wondered if people would feel too silly making all the ooooeeeeeuuuuuaahh’s for Vocal Joystick to be useful.
“As soon as we built the first prototype and I tried it, I immediately forgot that I was making sounds,” he said. “As soon as you start doing it, and see it respond to your voice, something amazing happens.”
On the Net:
The Vocal Joystick team plans to make the free program available at:
https://ssli.ee.washington.edu/vj/
© Copyright 2005 by DiverseEducation.com
