Read my lips. A new invention can recognise 鈥渟ilent speech鈥� by keeping tabs on your tongue and ears.
By training it to recognise useful phrases, it could allow people who are disabled or work in loud environments to quietly control wearable devices.
Advertisement
The new device relies in part on a magnetic tongue control system, previously designed to help people with paralysis drive a power wheelchair via tongue movements.
But the researchers were concerned that the technology 鈥� which relies on a magnetic tongue piercing or a sensor affixed to the tongue 鈥� might be too invasive for some users.
a professor at the Georgia Institute of Technology and technical lead on the wearable computer Google Glass, was inspired to try tracking ear movements after a dentist鈥檚 appointment. The dentist stuck a finger in Starner鈥檚 ear and asked him to bite down, a quick test for jaw function. As his jaw moved, so too did the space in his ears.
鈥淚 said, well, that鈥檚 cool. I wonder if we can do silent speech recognition with that?鈥� says Starner.
Infrared ear sensor
The resulting device combines tongue control with earpieces that look somewhat like headphones. Each is embedded with a proximity sensor that uses infrared light to map the changing shape of ear canal. Different words require different jaw movements, deforming the canal in slightly different ways.
As a test, the team listed 12 phrases that might be required, such as 鈥淚 need to use the bathroom鈥� or 鈥淕ive me my medicine, please鈥�. People were then recorded repeating these while wearing the device.
With both the tongue and ear trackers in, the software recognised what the wearer was saying 90 per cent of the time. Using ear trackers alone, the accuracy was slightly lower (IEEE Computer, DOI: 10.1109/MC.2015.310).
鈥楯补飞-别尘别蝉鈥�
The researchers hope to build up a phrasebook of useful words and sentences recognisable just from the ear data. 鈥淲e鈥檙e trying to figure out the fundamental parts of speech we can recognise. We call them 鈥榡aw-emes鈥�,鈥� says , a graduate student at Georgia Tech.
In addition, they鈥檝e started looking into other potential uses for the ear data. One experiment with a modified version of the ear trackers reached 96 per cent accuracy in recognising simple jaw gestures, like a move from left to right. Such gestures could let the wearer discreetly control a wearable device. Heartbeat monitoring also seems feasible, and could help the system verify it is placed correctly in the wearer鈥檚 ear.
works on silent speech in his lab at the Pierre and Marie Curie University in Paris. He says that demonstrating that the technology is 鈥渋ndustry ready鈥� will be key to bringing the technology to the market place.
鈥淭he true holy grail of silent speech is continuous speech recognition,鈥� says Denby, but the ability to recognise even a limited set of phrases is already a tremendous boon for some disabled individuals, he adds.
(Image: Ryan McVay/Getty)
