Anthropic has made a significant change to its model usage policy for the first time in a long time. The developers are now officially banning abusive and bullying behavior from their AI chatbot Claude, and will be able to terminate such chats early.
Anthropic has made a significant change to its model usage policy for the first time in a long time. The developers are now officially banning abusive and bullying behavior from their AI chatbot Claude, and will be able to terminate such chats early.
This is stated in the company’s official blog.
Under the new rules, Anthropic has banned “prolonged and gratuitous abusive or violent behavior” towards Claude. The main enforcement tool for this rule remains the chatbot’s right to prematurely terminate conversations with toxic users on the Claude.ai and Claude Code platforms. The company clarifies that this rule will only be applied in extreme cases and will not apply to ordinary user frustration, difficult creative topics, or testing models for robustness.
In addition to “model welfare,” Anthropic has significantly expanded its ban on military applications of its technology. The updated rules prohibit using Claude tocreate software for weapon control, guidance systems, and to arm drones and other autonomous vehicles. The developers admitted that they have recorded numerous attempts to use AI for such tasks.
The company also made it clearer that it would not allow people to be tracked without their consent or to create software to track them. This decision was made after a threat analysis report found cases of Claude-based systems being used to track political dissidents.
“Claude cannot be used to make decisions or make recommendations about who should be investigated, arrested, or charged in law enforcement or criminal proceedings,” Anthropic emphasized.
Other updates include a ban on covert information campaigns, voter disinformation, and account fraud. If an AI model is connected to autonomous physical equipment that could harm people, the policy requires the presence of a qualified operator for monitoring. Anthropic noted that the rules could be adjusted for individual government customers if the contract includes sufficient safeguards.
Unlike many other developers, Anthropic regularly explores the potential consciousness and moral status of AI. The company’s head of AI, Kyle Fish, and CEO Dario Amodei, have previously noted that questions about the internal experience of models become more important as they become more complex.
