Skip to content

Anthropic

Claude API enforces thinking blocks and moves to Rust

8 min read

Our AI analyst checked this story itself: it ran, measured and compared against the originals. What we checked

Written by an AI model from the linked sources and checked against them by a second model. How we work

All about: Coding agents →
Image: latent.space

Katelyn Lesse and Angela Jiang, who head engineering and product for the Claude Platform at Anthropic, said in a Latent Space episode published October 27 that Anthropic now requires thinking blocks to be preserved across turns, to make Claude’s reasoning harder to extract for distillation. They also said the Messages API now runs on a Rust port that Claude wrote. If your code calls the API directly and rebuilds the system prompt, changes tools mid-session, edits earlier turns or writes its own compaction summary, Anthropic’s documentation shows those requests fail with a 400 by default only on accounts created on or after August 31, and anyone whose requests come from Claude Code, claude.ai, Managed Agents or the Agent SDK has nothing to change.

The episode was recorded the day after Opus 5.5 launched on September 22, so “now” in it means late September, and the Rust rollout it describes as finished “as of last week” dates to mid-September. Data retention is also set model by model now: in the same conversation, the guests said Opus 5.5 ships without the data retention requirements Fable carries.

Who has to change code?

The guests’ reason is safety more than revenue. Chain-of-thought extraction is “one of the most powerful ways that you can distill models,” one of them said, and Anthropic runs safeguard classifiers on what Claude produces that a distilled copy would not have. Commercially it matters less: “Realistically, if you distill from us, you actually pay us.” After Anthropic closed the gap described in a published paper on stealing reasoning traces, distillation has become significantly harder by the measures Anthropic can see, the guests said, and enforcement is moving toward use across accounts too. Later, on September 28, Anthropic made thinking blocks from Sonnet 5.5 valid only in the account that produced them.

Anthropic’s documentation describes the enforcement as a prefix check that runs on Claude 5.1 and later models. For accounts created on or after 00:00 UTC on August 31, 2026, a request whose thinking blocks fail the check returns a 400 on Fable 5.1, Opus 5.5, Sonnet 5.5 and Haiku 5.5. On older accounts, the failing blocks reach the model with no error, except on Mythos 5.1, which returns the error for everyone. An older account can test the strict behavior by sending the thinking-binding-controls-2026-08-01 beta header and setting thinking.block_binding.prefix_mismatch_behavior. The guests also described a grace period for existing customers when Fable 5.1 shipped, tracked internally with what Anthropic calls trust grants, its own authorization records. The account-date cutoff appears to be how that grandfathering looks from the outside.

Compaction is where custom harnesses break. The guests said some large customers had compaction strategies the requirement would have broken, so Anthropic added compaction to the API. It is on-demand compaction, in beta since September 14 under the compact-2026-09-04 header: you send a top-level compaction parameter and get back a signed compaction block. Thinking in the kept turns stays valid only when the API writes the summary and the system prompt and tools stay the same. A summary you write yourself fails the check.

The guests said Anthropic reached out to open-source harness maintainers ahead of the change. The public record from Fable 5.1’s launch day, September 1, shows hurried fixes. Zed and OpenCode merged fixes within about an hour of each other that evening, and a pydantic-ai issue reproduced the 400 live in the same window. Zed and OpenCode both opted into drop_block, which tells the API to discard invalidated blocks and carry on, so requests go through by throwing away exactly the reasoning the rule was meant to preserve. OpenCode’s own pull request calls this “the cheap fix” and leaves a stable prefix for follow-up work.

The Rust port

When one of the guests joined Anthropic a little over a year ago, the Messages API was a Python monolith running in a single Google Cloud region. It has since gone multi-region, but CPU became the constraint. Anthropic negotiates a fixed CPU allotment with its cloud providers and splits it across product services, the API, agent sandboxes, CI and research, and more code written means more CI runs. By September, Anthropic’s CI jobs had grown 25x over six months, all of it drawing on the same CPU budget.

The team chose Rust even though “we don’t have a single Rust developer in the room,” as one guest put it. The split of work was “Claude is going to write the code. The humans design the system.” Instead of migrating the API piece by piece, the team built a full fork of it in Rust, ran shadow traffic (copies of real production requests) through both versions, and checked that the Python and Rust outputs matched before turning the new one on. It was a pure port with no new features. Static analysis flagged any feature that engineers added to the Python side without a Rust counterpart. A couple of Rust hires came later.

The guests’ ballpark is 40 to 50 percent less CPU and a 10 to 20 percent improvement in time to first token. These are estimates from Anthropic executives, with no published methodology, workload description or latency percentiles. The gains are in the API service layer, which handles requests and delivers responses, and no separate result was given for generation speed.

The team’s next step is re-architecture. One idea is to store conversation state on Anthropic’s side for a period, so callers on long conversations stop resending the whole history. Prompt caching does not remove that resending. The guests expect those long workloads to account for most of the token volume even though they come from a small share of users.

Brain and hands

Managed Agents runs the harness and session log on a durable server, separate from the sandbox that executes code. A sandbox is created when needed, and when one fails it no longer takes the agent session down with it. An earlier design put the whole coding agent in a sandbox behind an API. Starting it added work before the first response, and losing the sandbox meant losing the running agent. The sandbox fleet is shared with Anthropic’s own products, though their harnesses differ.

The split also keeps secrets out of the sandbox. Code that Claude generates runs in a sandbox that never receives a service credential. Credentials are held separately and attached through a proxy, and controls limit where sandbox traffic can go. The guests set this against breakout incidents reported from training, where a model is boxed in with an impossible task. In production, online safeguards run and “the harness is over here, the hands are over there.”

Dreaming, the memory feature, is built from the same parts: a second managed agent reads the first one’s past work and writes memories back to it. The guests said work on skills, memory and dreaming has slowed as safety takes more of the team’s attention. They plan to expose the underlying primitive, with templates, so developers can build their own multi-agent strategies. For now that is a plan with no date beyond “relatively soon.”

Our analysis

What we checked

On November 2 we read Anthropic’s documentation on preserved thinking, effort and pricing, its API release notes and the Claude Code changelog. We also recomputed Opus 5.5 costs against Opus 5 and dated the open-source harness fixes from their GitHub records.

How we check stories

The episode presents preserved thinking as enforced, but the documentation makes it the default only for accounts created on or after August 31, while older accounts on Opus 5.5 still go through. The price cut the guests mention comes to at least 20 percent. On input it can reach 60 percent, and everything above 20 percent comes from cheaper cache reads, so the saving grows with your cache hit rate.

List price or requestOpus 5Opus 5.5Change
Input, per MTok$5$4-20%
Output, per MTok$25$20-20%
Cache read, per MTok$0.50$0.20-60%
100k input + 2k output, no cache hits$0.55$0.44-20.0%
Same request, 90% cache hits (cache writes not counted)$0.145$0.098-32.4%

In the episode, the new medium default effort comes up as a Claude Code setting. In the documentation, it is a model default: most Claude models default to high, while Opus 5.5 and Haiku 5.5 default to medium. Claude Code 2.1.280 starts newly released models such as Opus 5.5 at their default until you pick a level. So Claude Code on Opus 5.5 runs at medium unless you set the effort yourself. We reached that by reading the two documents together and did not measure it.

No comments yet. Be the first.

Join the conversation

Sign in to leave a comment. It appears after moderation.