Editorial: The perils ahead for literate robots
A new breed of robot is using text-spotting software, dictionaries and internet access to learn to read anything, anywhere
INGMAR POSNER wants to develop robots that can see through walls. Not by equipping them with X-ray specs or specialised radar technology, but simply by teaching them to read.
Advertisement
鈥淏y reading a label on a closed door you can sometimes get a good idea of what can be found behind it,鈥� says , a roboticist at the University of Oxford. 鈥淩eading can help you detect things you cannot directly see.鈥�
Roboticists have spent years teaching their robots a range of skills to help them get by in the real world. Robots have learned to map their surroundings and to pick up and manipulate cumbersome objects. Some have even shown signs of becoming self-aware. But, remarkably, they remain illiterate.
With the written word so prevalent in the human world 鈥� from road signs to shop names 鈥� a non-reading robot trying to prove its worth is placed at a severe disadvantage, says , who works alongside Posner. Along with at the Queensland University of Technology in Brisbane, Australia, the team are trying to help robots level the playing field.
Teaching robots to read should, in principle, be relatively simple. After all, optical character recognition (OCR) software packages already exist. These automatically turn scanned images of books into text, and many researchers are using them to turn robots鈥� attention towards posters and signs on city streets. Last year, for example, Google launched , a smartphone application for just that task. Since May, Goggles has been able to , helping tourists work out what to order from a menu, for instance.
鈥淐urrent 鈥榬eading鈥� software tries to force everything a robot sees into text, such as walls and chimneys鈥�
Good OCR software is only a partial solution, however. Goggles relies on the user to recognise text and point a phone鈥檚 camera at the words before the OCR software kicks in. Robots will not have the luxury of human help, and researchers have found that OCR software cannot pick out words embedded in a busy scene by itself.
鈥淭he OCR software doesn鈥檛 cater for the fact that it might not be seeing text,鈥� says Posner. 鈥淚t tries its level best to force everything into text 鈥� brick walls, chimney stacks, everything.鈥� The result is a nonsensical muddle.
To get round this problem, the team developed text-spotting software. This relies on the fact that there is often a horizontal area of uniform colour just above and below text on a sign, but lots of two-tone colour variation within the text itself. Once the software has identified text, an image of it is passed onto the OCR software to read.
Even then, the results returned by the OCR software are often error-strewn. So the team has loaded their test robot, Marge, with a dictionary and spellchecker. This allows it to work out that 鈥渞oodbond鈥� is most likely a misreading of 鈥渂roadband鈥�, while 鈥渘qkio鈥� should be read as 鈥渘okia鈥�.
To understand names it reads in its environment, Marge turns to news websites, such as The New York Times and BBC Online. The robot trawls the sites for appearances of the word it has read, and analyses how often keywords like 鈥渞estaurant鈥� or 鈥渂ank鈥� appear in the same stories. This allows it to make strong semantic connections between frequent matches. Using this approach, Marge has learned that Strada is a UK restaurant chain, and that Barclays is a UK bank.
With those systems in place, Marge is now ready to read and exploit text in the world in the same way a human does 鈥� a 鈥渟eriously exciting鈥� prospect, says Posner. The work was presented at the in Taipei, Taiwan, last month.
One potential problem is identifying words that are difficult to read because of the viewing perspective, says at the University of Bristol, UK, whose team has developed its own software to help robots to read. Words printed on a curved surface can appear distorted, making them tricky for a robot to understand. Mirmehdi鈥檚 team is working on improving their software to overcome this, so a humanoid robot with dextrous hands can manipulate objects 鈥� like cylindrical paint cans 鈥� to read them more easily.
Posner hopes his team鈥檚 work will allow mobile robots to carry out tasks more easily by following signs the same way a human can. For example, a search-and-rescue robot in a building wouldn鈥檛 need to gradually build its own map of the building 鈥� it could read any available signs to find its way around.
While such end results are still a long way off, other roboticists agree that reading projects are worth pursuing. The work is 鈥渞efreshingly original in the robotics context鈥�, says at McGill University in Montreal, Canada. 鈥淚 personally believe that exploiting OCR methods in a mobile robotics context makes a lot of sense,鈥� he adds. 鈥淚n fact, once you reflect on it, there is no doubt it will be highly useful.鈥�
