Instructing an AI chatbot to answer questions as if it were in the TV show Star Trek seems to improve its mathematical ability, although no one is exactly sure why.
People who use chatbots like ChatGPT have already recognised that the quality of outputs can be improved by asking the AI to adopt a certain persona, or by bribing or threatening it. Rather than use trial and error to discover which such prompts are most effective, and at software firm VMware in California turned to the large language models (LLMs) that power the chatbots. They used them to fine-tune human-created prompts and then rated their effectiveness at called the GSM8K benchmark.
The researchers gave 60 initial prompts to three LLMs: one developed by French firm Mistral, which has recently partnered with Microsoft, and two versions of Llama2, made by Meta. The AIs were then tasked with improving the wording of the messages to make them more effective.
Advertisement
For example, from an initial prompt of 鈥淵ou are an expert mathematician. Solve the following math problem. Take a deep breath and think carefully,鈥� the AI-improved prompt might add requirements to define any assumptions, or to flag any loopholes used.
Those resulting prompts were then fed back into the AIs in an effort to tackle the GSM8K questions, which require simple arithmetic to solve, but take between two and eight steps to complete.
In nearly all instances, the AI models produced prompts that generated more correct answers to the GSM8K questions than human-created prompts. 鈥淚n my opinion, nobody should ever attempt to hand-write a prompt again,鈥� says Battle. 鈥淟et the model do it for you.鈥�
However, entrusting a chatbot to write prompts to help the same chatbot answer maths questions can result in some unusual exhortations. The highest-scoring prompt generated by the Llama2-70B model, for instance, asked the chatbot to adopt the persona of the captain of a Star Trek spaceship, jotting down answers in its 鈥淐aptain鈥檚 Log鈥� 鈥� something generated entirely spontaneously by the AI and not suggested by the initial prompts.
Why the AI produced such unusual prompts is 鈥渢he $64 million question鈥�, says Battle. 鈥淭o a certain extent, the answer is 鈥業 don鈥檛 care, just give the model what it wants.鈥欌�� However, thinking more scientifically, Battle believes it is a product of the data the model was trained on, perhaps with Star Trek content appearing more often with correct information. 鈥淲ho knows? There鈥檚 a lot of Star Trek references on the internet.鈥�
鈥淭he key thing to remember from the beginning is that these models are black boxes,鈥� says at Staffordshire University, UK. 鈥淲e won鈥檛 ever know why they do what they do because ultimately they are a melange of weights and probabilities and at the end a result is spat out.鈥�
However, despite the fact that the Star Trek-themed prompt was the most successful, both Flick and Battle say that you shouldn鈥檛 be addressing ChatGPT or other chatbots as 鈥渃ommander鈥� any time soon.
鈥淥ne thing is for sure: the model is not a Trekkie,鈥� says Flick. 鈥淚t doesn鈥檛 鈥榰nderstand鈥� anything better or worse when preloaded with the prompt, it just accesses a different set of weights and probabilities for acceptability of the outputs than it does with the other prompts.鈥�
Instead, the response to the discovery tells us more about ourselves and our perceptions of AI than the AI鈥檚 performance, believes Flick. 鈥淚t shows a fun result, but it鈥檚 a coincidental one that just so happens to match the desire for LLMs to be science fiction brought to life,鈥� she says.
arxiv
