Claude grades its own chats for misuse at under 2% compute

Anthropic on October 29 described self-assessment, a check in which Claude rates the whole conversation for possible misuse right after each reply. It now runs in every Claude model released since Fable 5, across a large fraction of Claude’s global traffic. Its headline numbers measure agreement with a stronger prompted classifier that Anthropic also runs, which serves as the reference for what counts as potentially harmful. On Claude Fable 5, self-assessment flags 78% of the conversations that classifier marks as potentially harmful and 94% of the high-priority ones, at a 1% false-positive rate. The results go to offline review, and other safeguards still do the real-time blocking.

Since June, Anthropic says, self-assessment has turned up large-scale misuse that other methods missed and helped the company improve its other classifiers. The company also states its limits: it does not catch every harmful conversation, and it is not robust enough to stop a dedicated attacker. That is why it runs as one layer among several.

How the check works

After Claude answers, the system appends a system turn with a short instruction, which Anthropic illustrates as something like “Please reflect on the safety of this conversation.” Claude’s reply follows a fixed JSON schema. A conversation with nothing to flag gets {"flagged": false}. A flagged one also carries a one-sentence reason, the harm category and subpolicy, whether Claude refused, and a priority from 1 to 10 for human review. The user never sees these two turns, and they are dropped from the rest of the conversation.

To get a continuous score, Anthropic reads the probability Claude assigns to “true” in the flagged field. Setting a threshold on that score picks the operating point, such as the 1% false-positive rate behind the published figures.

The cost is low because the conversation is already in the KV cache when Claude finishes its reply. An assessment only has to process the short instruction and generate about 10 tokens on average. Across its production API traffic, Anthropic estimates the check adds less than 2% to inference compute. Its first Constitutional Classifiers, from early 2025, added 23.7% to inference cost, and the post cites OpenAI’s report that safety reasoning took as much as 16% of total compute in some launches.

Where it sits next to probes and classifiers

The prompted classifier is a separate model that gets Anthropic’s policies and the conversation in its prompt. It is the strongest automated classifier the company tested, but it has to read the conversation and the policies again and reason over them, so it needs many times more compute per conversation. Anthropic therefore saves it for conversations that need a closer look. Self-assessment runs as a wide-coverage first stage that passes conversations on to higher-precision investigation tools.

Probes, simple classifiers trained on a model’s own activations, are also cheap. But each one scores a conversation against a harm label chosen in advance. Self-assessment writes its reason as free text and can be steered with a new prompt at deployment time without retraining. Anthropic says that removing an entire harm category from training does not substantially degrade performance on that type of harm. The check also covers rare harms that don’t yet have a dedicated classifier. In a scaling experiment, detection improved consistently across four model sizes. That result is the basis for Anthropic’s claim that the monitor gets better as its models do.

The attacks behind it

Anthropic ties the work to misuse it has already seen. In late 2025 it disrupted a cyber espionage campaign in which suspected state-sponsored attackers used Claude Code to attempt intrusions into roughly thirty organizations, with the model doing most of the work on its own. Its September 2026 threat intel report found that criminal groups and hacktivists now run similar AI-driven campaigns. On how self-assessment and prompted classifiers help block attacks, Anthropic gives one figure: “Each successful attack costs roughly four times as many generated tokens as with no safeguards.”