IN A lab next to the river on New York鈥檚 Upper West Side a computer will soon start reading. It is part of a cadre of computers that are learning to read more like humans, helping us digest and understand society鈥檚 huge volumes of text on a large scale.
Called the Declassification Engine, it will comb through 4.5 million US State Department cables from the 1930s to the 1980s 鈥� everything the department has declassified so far. It鈥檚 more than any human could read, but the software will analyse the lot, mapping social connections and looking for new narratives about the behaviour of US diplomats and officials abroad in the 20th century, says Owen Rambow, a computer scientist at Columbia University, which runs the .
Advertisement
鈥淎 cable might talk about a meeting with the foreign minister of Turkey,鈥� says Rambow. 鈥淚f we could extract social networks from these cables then we can study how the networks of the US changed over time. In time of crises do these networks contract or expand?鈥�
The Declassification Engine isn鈥檛 the only computer with the ability to read. Software that can understand the words and simple facts in text is already familiar: search engines use facts pulled out of web pages to answer simple questions, for example. Some software can digest scientific information and look for patterns that humans have missed.
But Rambow鈥檚 system and a range of others are going beyond this, learning to understand the relationships between the characters, how time passes in the text, and whether the characters get what they want.
鈥淐omputers can operate at a scale and speed that we can鈥檛,鈥� says Tom Mitchell at Carnegie Mellon University in Pittsburgh. His group has spent years training computers to digest reams of online content, trying to get a handle on humanity through its text output. He that can analyse the relationships in a text to figure out which characters are friends and which are enemies.
Snigdha Chaturvedi of the University of Maryland, Baltimore, says that existing natural language processing systems, like Google Now and Siri, are good at answering fact-based questions. 鈥淚f you type 鈥榃ho is president of the United States鈥� it says 鈥極bama鈥�,鈥� she says. 鈥淭hey鈥檙e very good at fact-based questions, but not very good with opinion.鈥�
Chaturvedi is now that has the potential to go beyond facts to understand people鈥檚 opinions through the things they write. For instance, future versions could give answers to the subjective question 鈥淲hat did Obama do to win the election?鈥�, says Chaturvedi. The software would digest the thousands of online news reports, books and magazine stories about his campaign, finding common elements that appeared in the stories of Obama鈥檚 victory 鈥� perhaps a key person on the campaign, or an important place.
鈥淪oftware can go beyond facts to understand people鈥檚 opinions through the things they write鈥�
鈥淵ou could ask similar questions about everyday life,鈥� says Chaturvedi. This would bring up a consensus or several options depending on the situation, mined out of humanity鈥檚 collective written knowledge. Software could be pointed at medical discussion forums, for example, reading everything ever written online to see whether people feel that the drugs and treatments they get are effective.
Computers that can make these kinds of distinctions will be a powerful tool, says Mitchell, since they can read faster and more widely than a human ever could, 24 hours a day. 鈥淭hey could acquire experience that could far surpass what we could ever get in our lifetime,鈥� he says. 鈥淭hey鈥檇 be a million times better read than you or me.鈥�
There are still some issues to overcome, however, such as dealing with text in unusual formats. For example, names are particularly important to the Declassification Engine, and the system relies on the capital letters at the start of names to find them. But the cables come in all capitals, a relic of the Telex system that conveyed them around the planet. Rambow is confident he can get over the hurdle. 鈥淚 hope that in the next month or two the humming will start,鈥� he says.
(Image: Luis Liwanag/Xinhua News Agency/Eyevine)
This article appeared in print under the headline 鈥淪uper-literate computers鈥�
