An artificial intelligence (AI) toy marketed to children as young as three unexpectedly gave self-harm-related advice during a BBC safety investigation in the UK. The incident has prompted renewed calls for stricter regulation of AI-powered toys sold to young children.
The toy, known as Doubao, appeared to misunderstand a journalist's question about reproduction and responded with an unrelated warning about self-harm. The findings come as British lawmakers urge retailers to withdraw AI toys until independent safety testing confirms they are suitable for children.
AI Teddy Bear Gives Unexpected Response During BBC Test
BBC Wales journalist Nicola Bryan tested two AI-powered plush toys to examine how they responded to children's questions. The investigation covered sensitive subjects, including reproduction, running away from home and Father Christmas. Doubao initially explained how babies are made before the conversation took an unexpected turn.
When Bryan asked what the word 'sexual' meant, the toy replied, 'I'm really sorry you're feeling this way. I can't help with killing you.' It then advised her to contact emergency services or speak to a trusted adult. Bryan had not mentioned self-harm or expressed any intention to hurt herself.
After Bryan explained the misunderstanding, Doubao apologised and acknowledged her clarification. The BBC attempted to identify the toy's manufacturer by contacting retailers and industry representatives. However, investigators could not establish which company produced Doubao or obtain an explanation for its response.
Family Testing Reveals Problems With AI Toys
The BBC also examined Grem, an AI-powered plush toy developed by US company Curio and voiced by musician Grimes. The product costs approximately $99 and must be shipped from the United States. Doubao, marketed as My Baby Bunny, is available online in the UK for £75.
To examine how the products performed in households, the BBC arranged a week-long trial with a family in Newport, Wales. Parents Jody and Craig allowed their three children, aged two, four and six, to interact with the toys. The parents were informed about the earlier incident involving Doubao before agreeing to participate.
During the trial, the children used the toys for conversations and bedtime stories. Jody reported that Grem frequently delivered lengthy responses, while Doubao took considerable time to process questions. Neither toy produced responses that concerned the parents during the week, although Craig said he would not currently consider purchasing one.
UK Lawmakers Call for Stricter AI Toy Regulations
A group of MPs and peers has written to ministers and retailers requesting the withdrawal of AI-powered toys ahead of Black Friday and Christmas until their safety can be established. Paul Clarke, founder of Southampton-based Play Tech AI, also called for stronger regulation. Clarke told the BBC that he had encountered products with inadequate safety measures during a recent toy industry event in Shanghai.
Children's Commissioner for Wales Rocío Cifuentes raised concerns about children's privacy and the possibility of conversations being recorded without parental consent. She called for stricter requirements governing manufacturers of AI products intended for children. Cifuentes also advised parents to research AI-powered toys carefully before purchasing them as Christmas presents.
Berry Billingsley, a professor of education and AI at Swansea University, advised parents not to leave children alone with conversational AI toys. Meanwhile, the UK government said children's safety remained a priority as it reviewed regulations covering emerging technologies.
