ChatGPT will end violent conversations and restrict weapon information for teens

OpenAI is tightening ChatGPT’s safeguards around violence and weapons, with stricter rules for teenagers and new interventions for conversations that show warning signs. The protections are expected to begin rolling out over the next few weeks.

Users identified as minors will receive less practical information about weapons and security systems even when a request appears harmless. OpenAI says its age-prediction system will keep reassessing signals as people use ChatGPT and move likely minors into the under-18 experience. Adults placed there incorrectly will be able to verify their age through Persona.

ChatGPT will also respond more cautiously when recent conversations suggest possible violence. If a user appears fixated on seriously harming someone, the system may automatically end the chat; repeated endings can lead to a temporary account pause. OpenAI is also limiting detailed reconstructions of mass attacks for all users.

The company says it made more than 200 referrals involving imminent threats to law enforcement in one recent quarter. It also committed to a twice-yearly Community Safety Report covering how harmful activity is detected, reviewed and escalated. The first edition will include anonymized examples, data and an assessment of the safeguards’ effectiveness.