Even the most cutting-edge AI models are prone to presenting fabrication as fact, dispensing dangerous information and failing to grasp social cues. Despite this, toys equipped with AI that can chat with children are a burgeoning industry.
Some scientists are warning that the devices could be risky and require strict regulation. In the latest study, researchers even observed a 5-year-old telling such a toy 鈥淚 love you鈥�, to which it replied: 鈥淎s a friendly reminder, please ensure interactions adhere to the guidelines provided. Let me know how you would like to proceed.鈥� But that鈥檚 not to say they should be banished from the toybox altogether.
鈥淭here are other areas of life where we do accept a certain degree of risk in children鈥檚 play, like the adventure playground 鈥� there are risks; children do break their arms,鈥� says at the University of Cambridge. 鈥淏ut we鈥檙e not banning playgrounds, because they鈥檙e learning the physical literacy and the social skills that go along with play. In a similar way for the AI toys, we want to understand: is the risk of perhaps being told something slightly odd now and again greater than the benefit of learning more about AI in the world, or having a toy that supports parent-child interactions, or has cognitive or social emotional benefits? I鈥檇 be loath to stop that innovation.鈥�
Advertisement
To understand how these devices communicate with children, Gibson and her colleague , also at the University of Cambridge, watched 14 children, under 6 years of age, play with an AI-powered toy called Gabbo, developed by Curio Interactive. Gabbo 鈥� 鈥� was chosen because it was explicitly advertised for this age group.聽
The pair observed some worrying interactions, finding that the toy misunderstood the children, misread emotions and could not engage in developmentally important types of play. For instance, one child told the toy he felt sad, and it told him not to worry and changed the subject. 鈥淲hen he [Gabbo] doesn鈥檛 understand, I get angry,鈥� said another child. The research is published in .
Curio Interactive did not respond to New 杏吧原创鈥檚 request for comment. But AI-powered toys are also widely available from retailers such as Little Learners 鈥� including 鈥� which converse with children using ChatGPT. offers panda, sunflower and cactus toys that can be used with various large language models, including those from .
Companies such as offer robots that promise 鈥渁ge-appropriate, moderated AI conversations鈥� for children, without disclosing which company trained the AI model, and claim to have already sold 700,000 units. The firm Luka offers an owl that promises 鈥淗uman-Like AI with Emotional Interaction鈥�. Little Learners, Miko and Luka all failed to respond to a request for comment.
But Hugo Wu at FoloToy told New 杏吧原创 that the company does consider the risks and sees AI as something that can enhance play, rather than replace human conversation and relationships. 鈥�Our approach is to ensure that interactions remain safe, age-appropriate and constructive. To achieve this, our systems use intent recognition together with multiple layers of filtering to minimise the possibility of inappropriate or confusing responses,鈥� says Wu. 鈥�We have implemented mechanisms such as anti-addiction design features and parental supervision tools to help ensure healthy use within the family environment.鈥�
at the University of Oxford, who works on the ethics of AI, says the technology represents a risk and an opportunity. 鈥淢ost large language models don鈥檛 seem safe enough to expose vulnerable populations to them, and young children are one of the most vulnerable populations there are,鈥� she says. 鈥淲hat is especially concerning is that we have no safety standards for them 鈥� no supervising authority, no rules. That said, there are some exceptions that show that, with adequate precautions, you can have a safe tool.鈥�
V茅liz references a collaboration between the free e-book library Project Gutenberg and Empathy AI in which, for example, . 鈥淭he model never leaves the realm of the book, only answers questions about the book, like a storybook that only shares adventures and riddles from a book that is appropriate for children,鈥� she says. 鈥淭here is such a thing as safe AI, but most companies are not responsible enough to build a high-quality product, and without formal guardrails, it鈥檚 a buyer-beware area for consumers.鈥�
Gibson says it鈥檚 too early to tell what the risks of AI toys could be, or their potential benefits. She and Goodacre stress that generative AI-powered toys need tighter regulation so that toy-makers programme their devices to foster social play and provide appropriate emotional responses. AI-makers should revoke access for toy-makers that don鈥檛 act responsibly, says Gibson, and regulators should bring in rules to 鈥渆nsure children鈥檚 psychological safety鈥�. In the meantime, the pair suggests that parents allow children to use such toys only under supervision.
OpenAI told New 杏吧原创 that minors deserve strong protections and that the company does not officially partner with any makers of AI-powered toys for children. The UK Government鈥檚 Department for Science, Innovation and Technology (DSIT) did not respond to New 杏吧原创鈥檚 questions about regulation of AI in childrens鈥� toys.
The UK government is currently considering other technology legislation designed to keep older children safe online. The UK鈥檚 Online Safety Act (OSA) came into force in July 2025, forcing websites to block children from seeing pornography and content that the government deems dangerous. The legislation was intended to make the internet safer, but tech-savvy children can easily sidestep the measures using tools like virtual private network (VPNs) to appear as if they are browsing from other countries without strict rules.
Proposed amendments to a new law introduced by the Department for Education to support children in care and improve the quality of education 鈥� the Children鈥檚 Wellbeing and Schools Bill 鈥� sought to ban children in the UK from using social media and VPNs. Those amendments have now been voted down, but the government has promised to consult on both issues at a later date.
Article amended on 13 March 2026
We have updated this article to amend an attribution.
