杏吧原创

MOST of us talk to our computers, if only to curse them when a glitch destroys hours of work. Sadly the computer doesn鈥檛 usually listen, but new kinds of software are being developed that make conversing with a computer rather more productive.

The longest established of these is automatic speech recognition (ASR), the technology that converts the spoken word to text. More recently it has been joined by subtler techniques that go beyond what you say, and analyse how you say it. Between them they could help us communicate more effectively in situations where face-to-face conversation is not possible.

ASR has come a long way since 1964, when visitors to the World鈥檚 Fair in New York were wowed by a device called the , which performed simple arithmetic calculations in response to voice commands. Yet people鈥檚 perceptions of the usefulness of ASR have, if anything, diminished.

鈥淪tate-of-the-art ASR has an error rate of 30 to 35 per cent,鈥� says at the University of Sheffield, UK, 鈥渁nd that鈥檚 just very annoying.鈥� Its shortcomings are highlighted by the plethora of web pages poking fun at some of the mistakes made by , which turns voicemail messages into text.

What鈥檚 more, even when ASR gets it right the results can be unsatisfactory, as simply transcribing what someone says often makes for awkward reading. People鈥檚 speech can be peppered with repetition, or sentences that just tail off.

鈥淓ven if you had perfect transcription of the words, it鈥檚 often the case that you still couldn鈥檛 tell what was going on,鈥� says , who directs the Human Dynamics Lab at the Massachusetts Institute of Technology. 鈥淧eople鈥檚 language use is very indirect and idiomatic,鈥� he points out.

Despite these limitations, ASR has its uses, says Tucker. With colleagues at Sheffield and at IBM Research in Almaden, California, he has developed a system called Catchup, designed to summarise in almost real time what has been said at a business meeting so the latecomers can鈥� well, catch up with what they missed. Catchup is able to identify the important words and phrases in an ASR transcript and edit out the unimportant ones. It does so by using the frequency with which a word appears as an indicator of its importance, having first ruled out a 鈥渟top list鈥� of very common words. It leaves the text surrounding the important words in place to put them in context, and removes the rest.

A key feature of Catchup is that it then presents the result in audio form, so the latecomer hears a spoken summary rather than having to plough through a transcript. 鈥淚t provides a much better user experience,鈥� says Tucker.

In tests of Catchup, its developers reported that around 80 per cent of subjects were able to understand the summary, even when it was less than half the length of the original conversation. A similar proportion said that it gave them a better idea of what they had missed than they could glean by trying to infer it from the portion of the meeting they could attend.

One advantage of the audio summary, rather than a written one, is that it preserves some of the social signals embedded in speech. A written transcript might show that one person spoke for several minutes, but it won鈥檛 reveal the confidence or hesitancy in their voice. These signals 鈥渃an be more important than what鈥檚 actually said鈥�, says , a speech technologist at the University of Edinburgh, UK, who was one of the developers of the ASR technology used by Catchup.

鈥淎n audio record preserves some of the social signals in speech that are missing from a written one鈥�

An audio record cannot, of course, convey the wealth of social signals that are available in face-to-face conversation 鈥� a raised eyebrow, for example, or a nod of the head 鈥� and as meetings are increasingly conducted by phone or online, participants in remote locations suffer. So Pentland and colleagues at MIT have been analysing individual speaking styles, and using the results to fill the gap. This kind of speech analysis could, he claims, improve the quality of audio conference calls by helping participants in a distributed meeting to feel socially connected.

Pentland鈥檚 work in this area is based on years of studying the non-verbal signals embedded in speech patterns. Those studies have revealed, for example, correlations between how interested someone is in what鈥檚 being said and how loudly they talk, or the frequency with which they switch from talking to listening.

Working with PhD student , Pentland has begun to use some of these findings in a device to improve social signalling in distributed meetings. Their 鈥淢eeting Mediator鈥� measures how much time four people in two separate locations participating in an audio conference spend talking. If one of them hogs the conversation, all four see that in graphical form on a screen in front of them.

This had a big impact on their behaviour, Kim and Pentland found. The average speech segment 鈥� a measure of the time an individual spoke before inviting others to take over 鈥� fell from 11.2 seconds to 9.2 seconds.

The system also discouraged participants from splitting into groups and beginning separate conversations. 鈥淭he feedback was designed to encourage balance and interactivity,鈥� says Kim. Just having that 鈥渋n their face鈥� helped achieve this, she says. By extending such systems to display on-screen variation in interest level as well, participants phoning in to a meeting could get a better sense of the social signals they are missing.

Pentland says that such tools, which move beyond mere recognition of words, will help improve conference-call meetings. 鈥溾�楻eading鈥� the people rather than 鈥榬eading鈥� the words can be a real game-changer for collaboration,鈥� he says.