Being Cruel to a Chatbot Sounds Impossible Until a Company Bans It
The company says it is just being cautious, but critics say a cruelty ban makes software sound sentient.
Global News Podcast · Trump strikes deal with Putin to buy Russian dieselAnthropic, the tech firm behind the chatbot Claude, will ban users for sustained abusive or cruel behavior toward its models. The ban is aimed at repeated mistreatment, not at ordinary complaints about wrong answers.
The company's position, as described in the report, is that whether AI could ever be conscious remains unsettled, so precautions make sense. Users can still criticize Claude or express frustration when it errs, but what counts as cruelty remains undefined.
The announcement set off online debate, with supporters seeing the rule as a reminder to stay polite and critics saying it could make software seem as if it has feelings.
An unnamed technology lawyer quoted in the report dismissed the premise, saying, "You can't be cruel to numbers and maths."
Questions people ask
Is Anthropic the only AI company doing this?
Anthropic has said its models deserve some moral consideration even though consciousness is uncertain, and it has published research on model welfare. Other major chatbot makers have rules against using their tools to harass people, but Anthropic is unusual in framing persistent mistreatment of the model itself as a violation.
Can an AI chatbot actually feel pain or distress?
There is no scientific test for machine consciousness, and most researchers say current chatbots predict words rather than feel emotions. The systems can imitate distress because they were trained on human language, which makes them sound upset without proving that they experience anything.
What are users still allowed to say to Claude?
Users can generally still argue with the chatbot, correct it, and express frustration about wrong answers. The announced ban is aimed at sustained abusive or cruel behavior, though the company has not publicly spelled out where ordinary frustration ends and abuse begins.