Anthropic bans ‘abusive or cruel behavior’ toward Claude

THE NEWS

What happened

Anthropic has updated its usage policy for Claude to explicitly ban sustained abusive or cruel behavior toward the AI model, framing it as a 'model welfare' issue. The update also strengthens restrictions on election interference, weapon development, and non-consensual surveillance.

CONTEXT

Why it matters

Anthropic has officially updated its usage policy for Claude, introducing significant new restrictions focused on 'model welfare' and high-risk misuse. The most notable change bans sustained abusive or cruel behavior toward the AI model itself. Additionally, the company has expanded its prohibition on weapon development to include software components that make weapons functional and actions like arming autonomous vehicles. The update also clarifies strict bans on non-consensual surveillance and election interference, including voter deception and impersonating officials. Anthropic states that terminating conversations remains the primary enforcement mechanism for these violations, though they did not confirm potential user bans. This marks a shift in how the industry views AI safety, moving beyond just preventing harmful outputs to protecting the model from mistreatment.

AT A GLANCE

Key facts

• Anthropic prohibits 'sustained and needless abusive or cruel behavior' toward Claude in its new usage policy.
• Terminating conversations remains the primary enforcement mechanism for violations of the model welfare policy.
• The ban on weapon development now explicitly includes software components that make weapons work and actions like arming drones.
• New rules strictly prohibit tracking people without consent and using the model to decide who to investigate or arrest.
• Anthropic notes it has seen multiple attempts to use Claude for developing guidance and control software for weapons.
• The policy update addresses election interference, including voter deception and impersonating candidates or officials.