Meta is introducing new safeguards designed to protect teenagers who may be experiencing emotional distress while interacting with Meta AI, including a system that alerts parents when conversations indicate possible risks of suicide or self-harm.

The move reflects growing concerns about the role artificial intelligence plays in young people’s lives and the need for stronger safeguards as teens increasingly turn to AI tools for information, support and conversation. Alongside parental notifications, Meta is also developing mechanisms that could enable emergency services to be alerted when users appear to be at immediate risk of taking their own lives.

According to the company, the latest measures are aimed at ensuring that teenagers receive age-appropriate AI experiences while encouraging them to seek support from trusted adults and professional services when dealing with serious emotional challenges.

Under the new system, when Meta AI detects signs that a teenager may be contemplating self-harm or suicide, it will continue directing the user to crisis helplines and encouraging them to contact a parent, counsellor or another trusted adult. In addition, Meta will now proactively notify parents who use Instagram’s parental supervision tools if a teen’s conversation with Meta AI suggests a potential risk.

The company said the detection system was developed with input from mental health experts and parents to identify conversations that may warrant intervention, including subtle references to self-harm.

Recognising the sensitivity of such alerts, Meta said every flagged conversation will undergo manual review before a notification is sent to a parent. The company added that where a teenager’s intentions are unclear, it will adopt a cautious approach and notify parents rather than risk overlooking a potential crisis.

Larry Magid, Chief Executive Officer and Co-Founder of ConnectSafely, welcomed the initiative, describing it as a balanced approach to protecting young people.

“While I believe that teens have a right to privacy, I also believe parents need to be informed if their teen may be at risk of hurting themselves,” Magid said.

“That’s why I advocated for this approach and support Meta’s decision to notify parents when, after careful review, it determines that a conversation with Meta AI contains indications of possible suicide or self-harm that warrant an alert. I appreciate how Meta struck the right balance; protecting teen privacy while ensuring parents have the information they need to support their teen.”

The parental notification feature has already been launched for Instagram supervision users in the United States, United Kingdom, Canada and Australia, with plans to expand availability globally before the end of the year.

The development builds on existing measures that alert parents when teenagers repeatedly search for suicide or self-harm-related content on Instagram within a short period.

Beyond parental notifications, Meta revealed it is developing systems that could allow emergency responders to be contacted when conversations with Meta AI suggest an individual may be at imminent risk of suicide.

The initiative extends practices already in place across Facebook and Instagram, where the company refers credible suicide-risk cases to emergency services. Meta disclosed that it made more than 19,000 such referrals globally last year, enabling first responders to conduct welfare checks on individuals believed to be at risk.

To strengthen how Meta AI responds to conversations involving suicide and self-harm, the company also conducted an extensive review process involving more than 75 mental health clinicians specialising in adolescent wellbeing.

The clinicians assessed hundreds of AI-generated responses to determine whether they were appropriate for teenagers, offering recommendations on how the system could better support vulnerable users.

Meta said insights from the review are already being used to improve AI interactions, particularly by ensuring responses acknowledge teenagers’ emotions while guiding them toward professional support instead of ending conversations abruptly.

Dr. Ji-yeon Lee, a licensed psychologist in the United States and Professor of Counselling Psychology at Hankuk University of Foreign Studies in South Korea, described the review process as unusually thorough.

“I was struck by the rigor of Meta’s clinical review process,” Lee said.

“It examined not only immediate responses to suicide and self-harm concerns, but also the broader conversational context, appropriate follow-up, and the varying levels of risk that can exist even within high-risk situations. This kind of expert-informed, scenario-based refinement is essential to making AI experiences safer for teens.”

The latest measures come as technology companies face increasing scrutiny over how AI systems interact with younger users, particularly around mental health, online safety and emotional wellbeing. For Meta, the new safeguards represent an effort to balance privacy, parental involvement and early intervention in situations where vulnerable teenagers may need urgent support.

Launching the Stricter Limited Content Setting for Meta AI Chats

Teen Accounts are automatically placed into a 13+ content setting, which also applies to conversations with Meta AI. In this default 13+ setting, Meta AI is designed to give age-appropriate responses and not comply with sensitive prompts. For example, Meta AI is trained not to engage in sexual or romantic conversations with teens or provide recipes for alcoholic drinks, directing teens to safer topics instead.

In October, we announced Limited Content as a stricter content setting for parents who prefer a more restrictive experience for their teens on Instagram — and now that stricter setting will apply to AI experiences too. This means that when parents opt their teens into the Limited Content setting on Instagram, it will also further limit the kinds of conversations teens can have with Meta AI. In the Limited Content setting, Meta AI will decline to respond to a broader range of prompts, further reducing the chance of potentially inappropriate conversations.