Google has launched a new AI model, dubbed Gemini, which it claims can outperform both 翱辫别苍础滨鈥檚 GPT-4 model and 鈥渆xpert level鈥� humans in a range of intelligence tests.
The firm鈥檚 CEO, Sundar Pichai, at Google鈥檚 I/O conference in May this year, although it was still in training at the time. But today the company has announced that it will be launching the cutting-edge model to the public.
Three versions of Gemini have been created for different applications, called Nano, Pro and Ultra, which increase in size and capability. Google declined to answer questions on the size of Pro and Ultra, the number of parameters they include or the scale or source of their training data. But its smallest version, Nano, which is designed to run locally on smartphones, is actually two models: one for slower phones that has 1.8 billion parameters and one for more powerful devices that has 3.25 billion parameters. Comparing the capabilities of AI models is an inexact science, but GPT-4 is rumoured to include up to 1.7 trillion parameters and Meta鈥檚 .
Advertisement
The mid-range Pro version of Gemini beats some other models, such as 翱辫别苍础滨鈥檚 GPT3.5, but the more powerful Ultra exceeds the capability of all existing AI models, Google claims. It scored 90 per cent on the , where an 鈥渆xpert level鈥� human is expected to achieve 89.8 per cent.
This is the first time an AI has beaten humans at the test, and is the highest score for any existing model. The test involves a broad range of tricky questions on topics including logical fallacies, moral problems in everyday scenarios, medical issues, economics and geography.
In the same test, GPT-4 scored 87 per cent, LLAMA-2 scored 68 per cent and Anthropic鈥檚 Claude 2 scored 78.5 per cent. Gemini beat all those models in eight out of nine other common benchmark tests.
The Pro model will be integrated into Google鈥檚 Bard, an online chatbot that was launched in March this year. The company says that another version of Bard called Bard Advanced will launch early next year and feature the larger Gemini Ultra model.
The new version of Bard will be available in English in more than 170 countries as of today, but it won鈥檛 be available in other languages or even in English across the UK and Europe. at Google says the delay is down to regulation rather than engineering: 鈥淲e鈥檙e working with local policies and regulators to make sure that we鈥檙e abiding by local laws and other such things before we launch in other areas.鈥�
at Google DeepMind says Gemini is the company鈥檚 largest and most capable model, but also its most general 鈥� meaning it is adaptable to a variety of tasks. Unlike many current models that focus on text, Gemini has been trained on text, images and sound and is claimed to be able to accept inputs and provide outputs in all those formats. But the Bard launch will only allow people to use text prompts as of today, with the company promising to allow audio and image interaction 鈥渋n coming months鈥�.
Collins says that Gemini is 鈥渟tate of the art in nearly every domain鈥� and that it is still in testing to determine exactly how capable it is at working in different mediums, languages and applications. 鈥淲e鈥檙e still working to understand all of Ultra鈥檚 novel capabilities,鈥� he says.
No versions of Gemini were made available for testing at the launch event, but Google showed demonstrations of the AI solving homework problems and working with live video input. It is also claimed to be better at developing software than previous models: last year, DeepMind released an AI-powered code generator called AlphaCode that the firm said could beat 50 per cent of human developers, and it is now releasing an updated version powered by Gemini that it claims can beat 85 per cent of human coders.
