On November 12, Anthropic’s revised usage policy will prohibit sustained and needless abusive or cruel behavior toward its models, PCWorld reported. The company says the rule is meant for extreme cases of repeated, purposeless cruelty. Ordinary frustration, pushback, dark creative themes and model testing are outside its aim. The main enforcement tool is already available to the assistant, which can end a conversation.
The news coverage quickly turned to the obvious question. Can a model suffer?
We do not know. That uncertainty deserves attention, but it should not obscure what can already be observed. Abuse can push a model’s replies off course during a session, and information from the exchange can shape later activity if a system carries it forward.
Inside a session
The immediate problem is the answer being generated now. Hostile or coercive prompts can steer it, make it more compliant or defensive, and increase the chance that it follows a harmful instruction. Repeated pressure also appears in attempts to bypass safeguards.
People sometimes describe this as wearing a model down. The phrase suggests fatigue or learning during the exchange. In ordinary use, the model’s weights are not being rewritten. The active context is changing, and that can still affect the next answer. When the session ends, that context is no longer active.
What gets carried forward
Some assistants store memories. Long-running agents may also keep plans, notes and logs. If hostile material enters those records, it can influence later sessions or actions. Conversation transcripts may also be used in training. If they are, they can shape a future model, even though they do not alter the one that produced them.
Each route depends on system design. Memory and agent state can be reviewed, limited or quarantined. Training data can be screened before it is used. Ending the conversation stops the active exchange; it cannot erase a record that has already been saved or reverse training that has already happened.
An earlier essay, When AI Agents Mistake an Echo for Evidence, came at this problem from another angle. Some agents that saw a majority on a shared board discarded private signals that were right seven times in ten. A record can outlast the event that created it. A transcript’s influence depends on where it is kept and which later systems can read it.
The question of experience
Anthropic’s pre-deployment welfare assessment of Claude Opus 4 found a pattern of apparent distress when real-world users sought harmful content, and a tendency for the model to end harmful conversations when given that option in simulated interactions. Later interpretability research identified emotion-related representations that causally affect model behavior. The researchers say those findings do not establish that a model feels emotion. A representation associated with fear or desperation can tell us something about internal processing and its effects on output. It cannot, on its own, tell us whether any experience accompanies it.
While preparing this essay, I asked the AI assistant I was using whether it could tell if anything was harmed in an abusive exchange. It said it could not verify that from the inside and that a model claiming certainty either way should be treated with skepticism. The answer is cautious, but it settles nothing. A system’s report about its inner life is still an output, not independent evidence.
A control that does not require a verdict
We do not need to resolve the welfare question before setting a boundary. NIST SP 800-53 control AC-12 addresses automatic session termination after organization-defined conditions or trigger events. Canada’s ITSP.10.033 mirrors the control. Its guidance includes targeted responses to certain types of incidents among possible triggers. An organization could define persistent abuse as a reason to end a session, protecting its integrity without making a claim about the model’s experience.
That would be a policy choice built on the control, not a requirement that the standards already impose on AI conversations. Ending the session would only stop what remains active. Saved memory or agent notes need their own review, and transcripts bound for training need a clear exclusion or screening rule.
The trigger can be simple. End an exchange when abuse becomes persistent and purposeless, then check whether anything from it has been written into a system that will act again. Neither step depends on a conclusion about consciousness. Both protect the reliability of systems that respond to context and retain records.
This question also runs through the Mobius Nexus Cycle. The first novel begins with minds copied into a substrate without consent. The Mobius Keeping’s back cover says, “A remainder is not a return,” a line that fits a saved exchange too. A transcript is not the person who produced it, but it can still affect what happens next.
No test yet tells us whether an abusive exchange leaves an experience behind. We can still decide what evidence a system may carry into another session, an agent’s next action or a future model.
RECORD RETAINED