DEAF people could soon receive subtitles for live events via their smartphone, thanks to a new online system for crowdsourcing captions.
Professional stenographers can provide real-time captions, but they cost up to $200 per hour and cannot be hired at the last minute or for short sessions, while automatic speech recognition systems have high error rates. So computer scientist at the University of Rochester, New York, and colleagues have turned to teams of non-expert workers who can be hired on demand and for much lower cost.
When users boot up the app, called , it connects to a central server and recruits workers through Amazon鈥檚 Mechanical Turk crowdsourcing service. Previous studies have shown that a willing group can be assembled within seconds. Each worker listens to a live audio stream from the user鈥檚 phone and attempts to write subtitles as quickly as possible.
Advertisement
All workers hear the full audio stream but the volume of different sections is raised and lowered, encouraging each person to focus on transcribing a particular part. Scribe then combines the partial transcriptions with software normally used to align evolutionarily related sequences of DNA. Bigham modified the software to account for common typos based on the layout of a keyboard 鈥� for example, if someone types 鈥渇qll鈥�, it is more likely they mean 鈥渇all鈥� than 鈥渇ill鈥�, because 鈥渁鈥� is nearer to 鈥渜鈥� than 鈥渋鈥� is. The software then chooses the words that a majority of the workers have typed.
In tests, Scribe was able to accurately capture 74 per cent of words spoken, compared with 88.5 per cent using professional transcription. 鈥淲e hope in the next few months to get a version of this out to deaf and hard of hearing students to better understand how they want to use the captions,鈥� says Bigham, who will present the work at the in Cambridge, Massachusetts, in October. Tabitha Allum, chief executive of UK charity , says Scribe could help meet an increasing need for captions. 鈥淎s our population gets older and people become more deaf, the demand for accessible talks and lectures is only going to get bigger,鈥� she says.
聯In tests, Scribe was able to accurately capture 74 per cent of words spoken聰
