Computers aren鈥檛 really known for their way with words, but that could be about to change. An artificial intelligence program recently scored as high as a 4-year-old on a test of verbal IQ. The result may help AIs develop common sense.
AIs such as Google鈥檚 search engine or IBM鈥檚 Watson typically perform well in specific areas, like ranking web pages or answering game-show style questions. But these systems tend to fail when asked to do things outside of their narrow area of expertise. For years researchers have attempted to build systems with a more general 鈥渃ommon sense鈥� understanding, but have had mixed results.
Step forward ConceptNet. Developed by Catherine Havasi and her team at the MIT Media Lab ConceptNet draws upon a crowdsourced database of millions of statements describing simple relationships between everyday objects, such as 鈥渁 fawn is a deer鈥� or 鈥渋ce cream is capable of melting鈥�.
Advertisement
Havasi describes the system as containing 鈥渢he kind of information that everybody knows about the world but that nobody ever writes down because we learn it too early鈥�.
To test ConceptNet鈥檚 overall intelligence, and Stellan Ohlsson of the University of Illinois at Chicago, who were not involved in the system鈥檚 creation, used a standard measure of child IQ called the The verbal portion of the test asks questions in five categories, ranging from simple vocabulary questions, like 鈥淲hat is a house?鈥�, to guessing an object from a number of clues such as 鈥淵ou can see through it. It is a square and can be opened. What is it?鈥�
To answer a question from the test, like 鈥淲hat do you wear on your head?鈥�, ConceptNet searches its database for the object that is most closely related to the pair 鈥渨ear鈥� and 鈥渉ead鈥�.
For the three main categories of questions鈥�� information, vocabulary and word reasoning鈥�� Sloan and Ohlsson found that the system鈥檚 aggregate verbal IQ was equal to that of an average human 4-year-old. 鈥淚 didn鈥檛 expect to see 4-year-old performance,鈥� says Sloan, who presented the results at the Association for the Advancement of Artificial Intelligence conference in Bellevue, Washington, last week. Havasi points out that this research only tested the system鈥檚 verbal ability and ignored parts of the test that covered spatial and symbolic reasoning.
She speculates that the latest version of ConceptNet, which has 17 million statements instead of the million available to the version tested, might score even higher. Sloan thinks that with better algorithms, ConceptNet could score as high as a 5 or 6-year-old.
