Editorial: 鈥Let the mind-reading begin鈥
Understand how the brain turns thoughts into words and you can make machines that read minds
AS YOU begin to read this article and your eyes follow the words across the page, you may be aware of a voice in your head silently muttering along. Most of us do it when we talk, listen, write letters or emails. The very same thing is happening to me as I type these words: a private internal narrative is shaping the words before I commit them to text.
Advertisement
What if we could tap into another person鈥檚 inner voice? Thinking of words does, after all, create characteristic electrical signals in our brains, and decoding them could make it possible to piece together our thoughts. Such an ability would have phenomenal prospects, not least for people unable to communicate as a result of brain damage. But it would also carry profoundly worrisome implications for the future of privacy. It鈥檚 time to start thinking about these things, because the technology to access those signals deep within the brain is fast becoming a reality. Welcome to the brave new world of mind-reading.
It would not be the first time that we have eavesdropped on the brain鈥檚 signals. The first scribbled records of electrical activity in the human brain were made in 1924 by a German doctor called Hans Berger using his new invention 鈥 the electroencephalograph (EEG). It uses electrodes placed on the skull to read the output of the brain鈥檚 billions of neurons.
By the mid-1990s, our ability to translate the brain鈥檚 activity into readable signals had advanced so far that people could move computer cursors using only the electrical fields created by their thoughts. The electrical impulses such innovations tap into are produced in a part of the brain called the motor cortex, which is responsible for muscle movement. To move a cursor on a screen, you do not think 鈥渕ove left鈥 in natural language. Instead, you imagine a specific motion like hitting a ball with a tennis racket. Training the machine to recognise which electrical signals correspond to your imagined movements, however, is time-consuming and difficult.
And while this method works well for directing objects on a screen, its drawbacks become apparent when you try using it to communicate. At best, you can use the cursor to select letters displayed on an on-screen keyboard. Even a practised mind would be lucky to write 15 words per minute with that approach. Speaking, we can manage 150.
Matching the speed at which we can think and talk would lead to devices that could instantly translate the electrical signals of your inner voice into sound produced by a speech synthesiser. Such an application would be a godsend for severely paralysed people, or those who are awake and fully conscious but are unable to communicate 鈥 so-called 鈥渓ocked-in syndrome鈥 鈥 who could once again engage in effortless conversations.
A new approach
This effortlessness, however, requires a significantly different approach. You need to focus only on the signals coming from the brain areas that govern speech. In 2008, neuroscientist Philip Kennedy at Neural Signals in Duluth, Georgia, tried to do just that by closing in on the small, face-specific part of the motor cortex, which controls the mouth, larynx and tongue. Here, it is possible to select only the muscle signals that are created when a person speaks.
For example, saying the word 鈥渂ee鈥 requires the mouth to execute very specific movements. First the lips must be pursed to form the initial 鈥渂uh鈥 sound, then the lips must stretch and the tongue press against the top row of molars to express the sound 鈥渆e鈥. All of these movements originate in the facial motor cortex, and so by tapping the signals before they mobilise their target muscles, you can work out what the person intends to say.
Kennedy and Frank Guenther, a neuroscientist at Boston University, read these signals in a person with locked-in syndrome by implanting penetrating electrodes in his facial motor cortex. It worked: they were able to identify three vowel sounds, and feed these signals to a speech synthesiser that broadcast the sounds in real time. The work was a first step in deriving the consonants and vowels that make up whole words. Kennedy thinks that by using this method he and Guenther could have a sample of 100 or more useful words for this patient within five years (PLoS ONE, vol 4, p 8218).
The method is not ideal. For one thing, it is specific to one patient. Perforating a person鈥檚 brain with an electrode is also not something to be undertaken lightly. And finally, while the move from using general motor signals to using language-specific signals is a significant step, it is not mind reading.
Real mind reading requires some way to intercept the signals before they hit the motor cortex. That means researchers have to search other parts of the brain for the source of language. Fortunately, they have a good idea where to look.
The translation of thoughts to language in the brain is an incredibly complex and largely mysterious process, but this much is known: before they end up in the motor cortex, thoughts destined to become spoken words pass through two 鈥渟taging areas鈥 associated with the perception and expression of speech.
The first is called Wernicke鈥檚 area, which deals with semantics 鈥 in this case ideas based more in pure meaning than in linguistics, which can include images, smells, or emotional memories (see diagram). Damage to Wernicke鈥檚 area can result in the loss of semantic associations: words can鈥檛 make sense when they are decoupled from their meaning. Suffer a stroke in that region, for example, and you will have trouble understanding not just what others are telling you, but what you yourself are thinking. Trying to speak results in garbled, nonsensical juxtapositions that neuroscientists call 鈥溾.
The second is called Broca鈥檚 area, agreed to be the brain鈥檚 speech-processing centre. Here semantics are translated into phonetics and, ultimately, word components. From here the assembled sentences take a quick trip to the motor cortex, which activates the muscles that will turn the desired words into speech. Injure Broca鈥檚 area, and though you might know what you want to say, you just can鈥檛 send those impulses, effectively rendering you mute.
When you listen to your inner voice, two things are happening. You 鈥渉ear鈥 yourself producing language in Wernicke鈥檚 area as you construct it in Broca鈥檚 area.
Outing your inner voice
The key to mind reading seems to lie in these two areas, but they had remained largely unexplored for good reason. Neuroscience was limited by the two main technologies that could read electrical signals in real time. Penetrating electrodes pick up with high fidelity the signals emitted by the few neurons they touch, but they inevitably carry risks.
Traditionally, the only alternative has been a surface EEG taken from electrodes sitting on top of the skull. Unfortunately, the bone blocks and jumbles the brain鈥檚 electrical signals, so while inexact signals can be used to deduce broad muscle movements, finding meaningful information in tiny areas like Broca鈥檚 and Wernicke鈥檚 requires much greater precision.
By 2004 neuroscientists had turned to a technique called electrocorticography (ECoG), which involves opening the skull and placing arrays of electrodes onto the top of the brain, just beneath the skull but without actually penetrating the brain (). ECoG has been used for several decades to locate the source of seizures in people with epilepsy.
Armed with ECoG, several researchers set about looking for the inner voice. Last year, Bradley Greger at the University of Utah in Salt Lake City successfully used ECoG to detect the electrical signatures of whole words 鈥 finer detail than anyone had ever obtained. By placing arrays of electrodes over both the facial motor cortex and Wernicke鈥檚 area, Greger and his colleagues detected the words 鈥測es鈥, 鈥渘o鈥, 鈥渉ot, 鈥渃old鈥, 鈥渢hirsty鈥, 鈥渉ungry鈥, 鈥渉ello鈥, 鈥済oodbye鈥, 鈥渕ore鈥 and 鈥渓ess鈥 as they were being produced in the brain of an awake, aware person with epilepsy.
Like Kennedy鈥檚 group, Greger says, 鈥渨e were going after a constant set of words鈥, specifically chosen to be useful to locked-in people. But unlike Kennedy, Greger was looking beyond the motor cortex. Though most of the signals still came from the facial motor cortex, this work nevertheless marked the first ever excursion into the brain鈥檚 language centres ().
Promising as it is, this approach does not scale up well. Like Kennedy鈥檚 word-mapping technique, it requires a new signal to be learned for each new word you add to the lexicon. English contains . And though this was the first instance of monitoring Wernicke鈥檚 area, it still relied largely on the facial motor cortex.
There might be another way. 鈥淵ou could go after more fundamental building blocks of language,鈥 says Greger. These building blocks are called phonemes and the English language has about 40 of them 鈥 the 鈥渒uh鈥 sound in 鈥渟chool鈥, for example, the 鈥渁h鈥 in 鈥渂lah鈥, the 鈥渟h鈥 in 鈥渟hy鈥. Every English word contains some subset of these sound components. Decode the brain signals that correspond to the phonemes, and you would have a kind of Rosetta Stone to unlock any conceivable word at the moment you think it.
Where would you find these phonemes? They would be much more likely to exist in Broca鈥檚 area or Wernicke鈥檚 area, the precursor regions to verbal motor signals.
In fact, tantalising clues from functional MRI scans had hinted at the presence of (Cerebral Cortex, vol 19, p 2156). So last year Eric Leuthardt at Washington University School of Medicine in St Louis, Missouri, and his colleague Gerwin Schalk positioned ECoG patches over the language regions of four fully conscious people. Sure enough, it was possible to detect the phonemes 鈥渙o鈥, 鈥渁h鈥, 鈥渆h鈥, and 鈥渆e鈥 ((Journal of Neural Engineering, vol 8, p 036004). The neural signatures of those sounds in the motor cortex looked the same whether the participants imagined they were speaking them or actually spoke them aloud. However, phonemes that were strictly imagined 鈥 sounds that the thinker was not intending to speak aloud 鈥 produced radically different signatures.
Leuthardt had been on the hunt for phonemes, but he had stumbled across something much better. Spoken phonemes activated both the language areas and the motor cortex. But imagined speech 鈥 that inner voice 鈥 boosted the activity of the neurons in the Wernicke鈥檚 area. 鈥淭his imagined speech, or inner voice, was especially easy to pick up in the classical language areas,鈥 he says.
鈥淭he imagined speech, or inner voice, was especially easy to pick up in the classical language areas鈥
鈥淧art of the way we process the world is through that internal dialogue,鈥 says Leuthardt. And he was able to listen in on it. He had effectively read their minds. 鈥淚 would call it brain reading,鈥 he says. 鈥淚t is literally taking your thoughts and translating them to the outside world.鈥 Leuthardt鈥檚 work has impressed other researchers. 鈥淭his is the first time anyone has been able to get these kinds of signals out of the language areas,鈥 says Greger. 鈥淚t鈥檚 really impressive.鈥
The real-time speed at which Leuthardt was able to detect the specific phonetic signals implies that it could be done as we think the words. We could then read signals off an ECoG at the instant a person thought them. 鈥淚t should be possible to produce real-time speech,鈥 Kennedy says.
To arrive at whole words, Leuthardt鈥檚 next step is to expand his library of sounds. He initially thought pursuing vowel phonemes would be far easier than consonant type phonemes because the mouth movements are so different, whereas transitioning between 鈥渒鈥 and 鈥渢鈥 barely causes the mouth to move. However, that is not necessarily a problem. It鈥檚 too early to discuss the research findings, but progress is happening faster than he had anticipated, and Leuthardt is very happy with the results. 鈥淲e鈥檙e burning through [phonemes] fast,鈥 he says.
For now, this research is primarily aimed at improving the lives of people with locked-in syndrome, but the ability to explore the brain鈥檚 language centres could revolutionise other fields. The consequences of these findings could ripple out to more general audiences, who might like to use extreme hands-free mobile communication technologies that can be manipulated by inner voice alone; consider a device that could someday give you your own private telepathic link. For linguists, it could provide previously unobtainable insights into the neural origins and structures of language. For example, linguists currently have only a hazy understanding of the neurobiological processes that create language from thought.
Leuthardt鈥檚 next project is a collaboration with international researchers to find out how the production of phonemes translates across different languages. If linguistic thoughts activate the same groups of neurons in people across cultures and languages, that might imply that humans are in some way hard-wired to communicate using these types of sounds.
Leuthardt is not, however, content to stop at the inner voice. He is also looking further, to see whether semantic understanding can be mapped between vastly different cultures: which groups of neurons fire, for example, at the idea of 鈥渄og鈥 as thought by an American and a Chinese person? 鈥淲e want to see how far we can reach,鈥 Leuthardt says. 鈥淟et鈥檚 see what semantic representations look like.鈥 If it were possible to generalise these semantic images and use them for communication instead of language, universal translators might be within reach. Indeed, knowing what someone is thinking without needing words at all would be functionally indistinguishable from telepathy.
Is the world ready for mind reading? Or is it the stuff of Orwellian nightmares 鈥 a technology that makes it possible to read your deepest, darkest secrets? Delve far enough into the brain鈥檚 idea-making processes, after all, and it should be possible to probe thoughts that were never intended for sharing. Where Kennedy鈥檚 method relied on speaking or the intent to speak, Leuthardt鈥檚 technique can pick up those thoughts without the participant ever intending to speak them.
For now, this difference is academic. This version of mind-reading requires brain surgery. But if less invasive techniques could harness the same information, all bets would be off. If that were to happen, it would be important to ensure that there was a mechanism to stop information being extracted without the subject鈥檚 cooperation.
In fact, such a mechanism already exists. If we always said every word we were thinking, the consequences would be disastrous. Instead, we separate public from private with a kind of self-censoring mechanism, though the point at which this mental brake comes into play during speech generation is unclear, says Leuthardt. It鈥檚 an important distinction, because if the brake exists only in the motor cortex, information accessed at the language centres would be unfiltered.
What do you think? Are we ready to breach the last bastion of privacy 鈥 our own skulls 鈥 to reveal the things we have so far kept safely locked away in the darker corners of our heads? No, don鈥檛 say anything. I鈥檝e already read your mind.
