Anthropic has introduced a new feature for its largest AI model, Claude, that allows it to stop communication in exceptional cases where users persistently try to sway the system towards harmful or dangerous topics.
Anthropic has introduced a new feature for its largest AI model, Claude, that allows it to stop communication in exceptional cases where users persistently try to sway the system towards harmful or dangerous topics.
According to Anthropic, the new option currently only works in Claude Opus 4 and Opus 4.1 models. This applies to situations where users repeatedly request the creation of prohibited content, for example, involving minors, or try to obtain instructions for organizing large-scale violence or terrorist attacks.
The company explains that the move is not so much about user safety as it is about the concept of «model welfare.» Anthropic doesn’t claim that the AI is conscious or sentient, but testing has shown that in similar situations, Claude has shown a «strong aversion» to dangerous requests and even «signs of stress.» That’s why the company decided to add a tool that allows the system to end a fruitless or harmful dialogue after several unsuccessful attempts to refuse the user.
It is noted that the ability to end a conversation is a last resort: Claude uses it only when there is no longer a chance for productive communication, or if the user himself asks to close the chat. In this case, the account is not blocked — you can start a new conversation or create a thread from the previous one. The model will also not end the dialogue in cases where a person may be at risk of suicide or harming others.
Anthropic has been experimenting with «ethical frameworks» for AI lately. The company emphasizes that the current changes are more of an experiment to test approaches that may be needed in the future. The developers see the feature as one of the «low-cost safeguards» that can reduce legal and societal risks if the question of the «moral status» of large language models ever becomes relevant.
As a reminder, we also published an article about how Anthropic made Claude’s learning mode available to all users: a chatbot explains its coding decisions. Since April, learning mode has been available only to Claude for Education users.
