Some artificial intelligences聽can perform tasks with superhuman ability, but just how clever are they overall? As smart as a honeybee? A Labrador? Or a chimp? A competition called the Animal-AI Olympics will pit AIs against tests normally used to study animal intelligence.
From April, AIs will battle it out聽in a virtual playground for a $10,000 prize pool. All the tasks involve retrieving a piece of food, but the skills needed to succeed vary in complexity. This mimics real-life experiments used to measure animal intelligence.
Entrants will complete tasks they haven鈥檛 seen before to eliminate the opportunity to swot up beforehand, says Matthew Crosby at the Leverhulme Centre for the Future of Intelligence, UK.
Advertisement
The Animal-AI Olympics will test a range of cognitive skills, such as the ability to reason, navigate and learn from past experiences. 鈥淥ne key concept is聽object permanence, the understanding that objects continue to exist even when they鈥檙e out of sight,鈥� says Crosby.
Although it won鈥檛 be in the competition, the A-not-B task is one such test of this ability. In the聽task, an animal is repeatedly presented with two cups, A and B. For the first few iterations, cup A always contains a piece of food.
However, once the animal is trained to understand this, the experimenter switches the food to cup B in plain sight. Some animals, such as dogs, continue to聽persevere with cup A, but others, such as macaques, instantly switch to cup B.
Tested to the limit
The variety of tasks in the competition will challenge one key limitation of many AIs: once they learn something, it is very difficult for them to adapt that knowledge to a similar but different situation. For example, one AI can outperform humans at聽the video game StarCraft and another beats us at the board game Go, but they are both useless at most other tasks unless聽completely retrained.
A will be provided during the competition, which runs until November.
The big hurdle will be to build AIs with general intelligence: systems with common sense and聽the ability to do a wide range of tasks based on limited data, says Chris Summerfield at the University of Oxford.
Many AI algorithms mimic certain functions in animal brains, such as the visual system in primates, he says. This is why image-recognition software, such as used by Google鈥檚 reverse image search, has been so successful.
But AIs lack many other brain features that contribute to cognitive ability, including short鈥憈erm memory or future planning. This may explain why AIs are good at specific tasks, but struggle to adapt to others.
Testing AI systems in unfamiliar environments is an important step to creating AIs that can solve a wide range of problems beyond those they were initially designed for, says Crosby.
鈥淲e expect this to be a hard challenge,鈥� he says. 鈥淎 perfect score will require a breakthrough in AI, well beyond current capabilities. However, even small聽successes will show that it聽is聽possible, not just to find useful聽patterns in data, but to extrapolate from these to an understanding of how the world聽works.鈥�
