- Meta is rolling out a new safety feature that will see parents alerted if their teenager discusses suicide or self-harm with Instagram's built-in AI chatbot.
- The tech giant has developed an AI tool that will flag conversations of concern for manual review before an alert is sent.
- When a teenager suggests they may be thinking about harming themselves, Meta AI directs them to crisis helplines and encourages them to speak to a trusted adult.
- If the intent is ambiguous, Meta says it will 'err on the side of caution and alert the parent'.
- 'While that means we may sometimes notify parents when there may not be real cause for concern, we feel this is the right starting point,' Meta added.
- The alerts are only available on accounts that are supervised by a parent or carer and are already live in the US, UK, Canada, and Australia.
- The feature has been developed after feedback from more than 75 clinicians.
- Meta is also building ways to alert emergency services if someone 'appears to be at imminent risk of suicide'.
- Parents will be signposted to expert resources to help them approach conversations about suicide and self-harm with teenagers.
- Instagram also alerts parents if a teen is repeatedly searching terms related to suicide and self-harm within a short space of time.
- Supervising parents can also see the topics teen users have asked Meta AI about in the past week.
Meta's new safety feature for Instagram aims to protect teenagers by alerting parents if their children discuss suicide or self-harm with the platform's AI chatbot. This initiative follows feedback from over 75 clinicians and is designed to foster open conversations about mental health.17
When a teenager indicates potential self-harm, the AI directs them to crisis helplines and encourages discussions with trusted adults. The alerts are only available on accounts supervised by parents or guardians and are currently live in the US, UK, Canada, and Australia, with plans to expand globally by year-end.36
Meta emphasizes a cautious approach, stating, “While that means we may sometimes notify parents when there may not be real cause for concern, we feel this is the right starting point.” The company is also developing methods to alert emergency services if a user appears to be at imminent risk of suicide.58
In addition to alerts, parents will have access to expert resources to facilitate discussions about these sensitive topics. The feature also monitors repeated searches related to suicide and self-harm, providing an additional layer of support for at-risk teens.911
Meta's commitment to improving AI interactions with teenagers on sensitive issues reflects a growing recognition of the need for mental health support in digital spaces.
“Developed after feedback from more than 75 clinicians, the feature is already live in the US, UK, Canada, and Australia, with a global rollout planned by year-end. Meta says it will 'err on the side of caution' when intent is ambiguous, potentially notifying parents without real cause for concern.”
