Meta is ramping up safety for teens to prove to parents and regulators that its platforms are safe. A few months ago, the company made it possible to see the topics of teens' conversations with AI, and now it's launching a new safety feature that will alert parents if their child's chats show signs of self-harm or suicidal thoughts.
Meta is ramping up safety for teens to prove to parents and regulators that its platforms are safe. A few months ago, the company made it possible to see the topics of teens' conversations with AI, and now it's launching a new safety feature that will alert parents if their child's chats show signs of self-harm or suicidal thoughts.
When a minor user talks about suicide or self-harm, Meta AI now directs them to support hotlines and encourages them to reach out to an adult who can help.
Starting today, the company will also proactively notify parents or supervising guardians that a teen is at risk, and Meta will share resources and tips with them on how they can help their children, Engadget reports .
Meta says it has created a special AI system to detect such conversations, but all chats recorded by the AI will first be manually checked. This will be done before sending a warning, so as not to spread panic without real reasons. At the same time, the company will be guided by the principle of maximum caution: notifications will be sent even when the teenager's intentions seem ambiguous to the moderator.
The feature is currently available to parents using Instagram parental controls in the US, UK, Australia and Canada, and will be available to all other parental control users around the world by the end of the year.
The company is also working on creating the ability to contact emergency services if a user's (both minor and adult) conversation with Meta AI indicates an immediate threat to their life.
Finally, Meta’s content restriction settings now apply to interactions with Meta AI. If parents have enabled Instagram’s content restriction feature, which filters out sensitive topics like explicit images or self-harm, their children’s interactions with the AI will also be more restricted.
