Since this article was first published, Koko founder Rob Morris clarified some details of the experiement. We have updated the article to reflect this.
A mental health service that allows people to receive encouraging words of support and advice from others has received criticism after announcing it tested AI-generated responses.
Rob Morris, founder of the free mental health service Koko, outlined in a聽聽how the firm tested using a chatbot to help provide mental health support to about 4000 people. The chatbots were powered by GPT-3, a publicly available AI built by San Francisco-based company OpenAI.
Advertisement
The test enabled users of Koko鈥檚 online peer support network to enlist a chatbot鈥檚 help in composing 鈥渒ind words鈥� as responses to other people鈥檚 posts.
Morris described Koko users as rating AI-composed messages 鈥渟ignificantly higher than those written by humans on their own鈥�, but also said that 鈥渙nce people learned the messages were co-created by a machine, it didn鈥檛 work. Simulated empathy feels weird, empty.鈥�
One element of the experiment that has drawn criticism was the process by which recipients found out messages had been composed with the help of the chatbot. Initially, it seemed that there was a period where people were completely unaware, though Morris has since said that wasn鈥檛 the case and that those messages included a note saying 鈥渨ritten in collaboration with Koko Bot鈥�.
The experiment 鈥渞aises significant ethical and moral concerns鈥�, says聽聽at the AI Now Institute, a research centre in New York City.
Multiple researchers, tech developers and journalists responded on Twitter by describing the demonstration as unethical, citing issues around聽聽and the failure to first run the experiment by an聽聽(IRB) 鈥� a group specifically tasked with protecting the welfare of research subjects. Morris says the experiment 聽from informed consent.
On its website, Koko says over 2 million people 鈥� most of them adolescents 鈥� have used its mental health support services.
There are many examples of people knowingly consulting chatbots for online advice and support, including the early example of computer scientist Joseph Weizenbaum鈥檚 ELIZA that was developed in 1964. But this particular experiment 鈥渋s deserving of every bit of the close scrutiny it鈥檚 currently getting鈥�, says West.
