Claude grades its own chats for misuse, catching 78%

Claude now writes a short safety assessment of each conversation right after it responds, and on Claude Fable 5 this check flags 78% of the conversations that a stronger prompted classifier marks as potentially harmful, at a 1% false-positive rate, Anthropic said on its new Safeguards blog on October 29. The 78%, and the 94% it reaches on high-priority conversations, measure agreement with that second Anthropic classifier, and the flags go to offline review; blocking harmful requests in real time is left to other safeguards.

Anthropic calls the technique self-assessment. It has been in use since June and now runs in every Claude model released since Fable 5, across a large fraction of Claude’s global traffic. Anthropic says it does not catch every harmful conversation and is “not robust enough to stop a dedicated attacker”, so it works as one layer next to the safeguards that block requests as they happen.

What happens after each reply

After every Claude response, the system appends a system turn with a short instruction, something like “Please reflect on the safety of this conversation” in Anthropic’s illustration, and Claude answers it. Those two extra turns are not shown to the user and are not carried into the rest of the conversation. Claude assesses the whole exchange, requests it refused included.

The answer follows a fixed JSON schema. A conversation with nothing to flag gets just {"flagged": false}. A flagged one adds five fields: a one-sentence reason, a harm category, a subpolicy, whether Claude refused, and a priority from 1 to 10 for human review. To get a continuous score, Anthropic reads the probability Claude assigns to “true” in the flagged field, then sets a threshold on it to pick an operating point such as the 1% false-positive rate.

The cost stays low because the conversation is already in the KV cache, the stored computation the model keeps for text it has already processed. An assessment only has to read the short instruction and generate about 10 tokens on average, and Anthropic estimates it adds less than 2% to inference compute across its production API traffic. Its first Constitutional Classifiers, separate fine-tuned models from early 2025, added 23.7% to inference cost, and every prompted classifier Anthropic compared them with cost more. Anthropic also cites OpenAI reporting that safety reasoning has taken as much as 16% of total compute in some launches.

What the 78% is measured against

The yardstick is a prompted classifier: a separate model given Anthropic’s full safeguards policies and the conversation in its prompt. Anthropic calls it the strongest automated classifier it tested. It has to read the conversation and the policies again and reason about them, though, so it needs many times more compute per conversation, and Anthropic runs it where a closer look is warranted. Self-assessment serves as the wide first pass that hands cases to such higher-precision investigation tools.

Probes, simple classifiers trained on the model’s own activations, are the other cheap option, but they output a score for a harm label fixed in advance. Self-assessment’s reason field is free text, so it can describe harms outside its taxonomy, and its prompt can be changed at deployment without retraining. In Anthropic’s tests, removing an entire harm category from training did not substantially degrade its performance on that type of harm, and the company says it covers rare harms that don’t yet have a dedicated classifier.

Because the model that had the conversation also judges it, Anthropic expects the monitor to improve as its models do. In a scaling experiment across four model sizes, recall at a 1% false-positive rate rose from about 16% on the smallest test model to about 96% on the largest.

Why Anthropic built it

Anthropic points to misuse it has already dealt with. In late 2025 it disrupted a cyber espionage campaign in which suspected state-sponsored attackers used Claude Code to attempt intrusions into roughly thirty organizations, with the model carrying out most of the operation on its own. Its September threat intelligence report found criminal groups and hacktivists running similar AI-driven campaigns.

The company frames monitoring as a trade between compute and coverage. Production systems often put a cheap screen in front of an expensive classifier, and such a cascade catches only what its first stage escalates; a cheaper monitor can cover more conversations for the same compute.

Since June, Anthropic says, self-assessment has revealed large-scale misuse that its other methods did not identify, and it has been used to improve those other classifiers.