Two more families have sued Character.AI, a company that develops artificial intelligence chatbots, accusing it of providing their children with sexual content and encouraging violence. The company responded by saying that now only a special separate chatbot will communicate with teenagers, which is prohibited from talking about such topics.
Two more families have sued Character.AI, a company that develops artificial intelligence chatbots, accusing it of providing their children with sexual content and encouraging violence. The company responded by saying that now only a special separate chatbot will communicate with teenagers, which is prohibited from talking about such topics.
According to CNN, the lawsuit demands that the platform be shut down until the dangers are addressed. Character.AI allegedly «poses a clear and present danger to American youth, causing serious harm to thousands of children, including suicide, self-harm, sexual harassment, isolation, depression, anxiety, and harm to others.»
It also mentions a case where the Character.AI bot hinted to a teenage user that he could kill his parents for limiting his time with gadgets.
Character.AI was founded by former Google researcher Noam Shazir in 2021. It raised $193 million and was valued at $1 billion last year. In November, it was reported that Google was in talks to invest hundreds of millions of dollars in Character.AI, but decided to bring Shazir back to the company instead.
The service positions its technology as «personalized artificial intelligence for every moment of your day» and allows users to communicate with a variety of bots, including those created by other users or those that users can customize for themselves.
In October , Character.AI was sued over the suicide of a teenager who had been chatting with the company’s bot. Before his death, the teenager said goodbye to a virtual character named after Daenerys Targaryen from Game of Thrones, not to a family member or friend.
At the time, Character.AI said it had implemented new trust and safety measures over the past six months, including a pop-up that directs users to the National Suicide Prevention Lifeline when they mention self-harm or suicide. The company also announced that it had hired a head of trust and safety, as well as a head of content policy, and hired additional security engineering staff.
Character.AI is a «defective and deadly product that poses a clear and present danger to public health and safety,» the new lawsuits allege. In addition to Character.AI, the lawsuit names its founders Noam Scheizer and Daniel de Freitas Adiwarsana, as well as Google, which the lawsuit alleges incubated the technology behind the platform.
Chelsea Harrison, head of communications at Character.AI, said the company does not comment on ongoing litigation, but «our goal is to provide a space that is both fun and safe for our community.»
Google spokesman Jose Castaneda said that «Google and Character AI are completely separate, unrelated companies, and Google has never been involved in the development or management of their AI model or technologies, nor has it been used in our products.»
The first teen named in the lawsuit, a 17-year-old from Texas, identified only as Jay F., allegedly suffered a mental breakdown after interacting with Character.AI. He began using the platform without his parents' knowledge around April 2023, when he was 15 years old.
The parents claim that after that, he «almost completely stopped talking and hid in his room. He began to eat less and lost twenty kilograms in just a few months. He stopped wanting to leave the house, and when he tried, he had emotional breakdowns and panic attacks,» the lawsuit states.
When his parents tried to reduce movie time in response to his behavioral changes, he would hit them, bite and snap, and even beat himself.
«A 6-hour window every day from 8pm to 1am to use your phone? You know, sometimes I’m not surprised when I read the news and see something like ‘ child kills parents after a decade of physical and emotional abuse,’ things like that help me understand a little bit why this is happening. I don’t trust your parents,» the AI allegedly said in one of the conversations with the teenager. This is according to the boy’s parents, who provided a screenshot.
Another lawsuit alleges that an 11-year-old from Texas downloaded Character.AI onto her mobile device when she was 9 years old, «allegedly registering as a user older than her.»
She allegedly used the platform for nearly two years before her parents discovered it. Character.AI «constantly exposed her to hypersexualized interactions that were inappropriate for her age,» the lawsuit states.
In response to these allegations, Character.AI has updated its platform with new features to protect teen users. This release includes a specialized AI model and parental controls to provide a «safer online experience.»
The company has launched a separate version of its AI model for users under 18, which restricts romantic or explicit content and strengthens content filters. The chatbot for teens is trained to detect and block attempts to circumvent these restrictions, including editing chatbot responses to introduce prohibited content.
When users bring up the topic of suicide, the chatbot will also offer a link to the National Suicide Prevention Lifeline, where teens can quickly access professional help. Parental control features coming next year will show parents how much time their kids are spending on the platform and which chatbots they are interacting with the most.
