“Verify twice” wastes tokens on Opus 5, Anthropic says

Anthropic has added three cost-cutting commands to its claude-api skill: /claude-api prompt-audit, /claude-api cost-optimize and /claude-api hillclimb. In a September 10 post on the Claude blog, the company published its own test results for each one. The biggest result came from hillclimb. It moved a customer support setup from Opus 4.8 at high effort to Sonnet 5 at low effort, and on held-out tickets it scored 90.5% against the original 78.6%, at about one fifth of the cost.

The claude-api skill ships with Claude Code. It is also in Anthropic’s open-source skills repository, so it can be installed anywhere Agent Skills run. The post covers three levers: the prompt cache, leftover instructions in prompts, and effort.

Where the cache breaks

Prompt caching stores the model’s processed state for a prompt prefix, so a later request that starts with the same bytes skips that work and pays a reduced read price. Three constraints decide whether it works: the cache is pinned to one model, matches must be byte-exact across the whole prefix, and entries expire. The 5-minute TTL counts from the start of the request. When an agent waits on a tool call or a subagent for longer than that, the next turn rewrites the cache at 1.25 times the normal input price, or 2 times for a 1-hour cache, when it could have paid the read price.

Most misses come from small things: a timestamp or ID in the system prompt, tool definitions that change order, or switching effort or thinking settings mid-conversation, because those settings are rendered ahead of your content. Opus 5 and Fable 5.1 are the exception and let you change effort without breaking the cache. Anthropic lists several fixes. Mark rarely used tools defer_loading so they stay out of the prefix until tool search pulls them in. Add system instructions as messages instead of editing the system prompt. Save model or effort switches for moments like compaction, when you pay for a miss anyway. To pre-warm the cache, send a request with max_tokens: 0 and an explicit breakpoint at session start, for example while the user is still typing. When hits drop, Claude Console’s cache diagnostics show where two consecutive requests diverged.

The audit on Opus 5

prompt-audit looks for instructions written to patch older models: “verify twice”, “be maximally thorough”, mandatory step procedures, scratchpad templates, few-shot examples tuned to old failure modes, contradictory rules and dated settings such as manual thinking budgets. It scans prompts, skills and tool descriptions in the working directory, including CLAUDE.md. The list follows Anthropic’s earlier advice for the Claude 5 generation to trust the model’s judgment over rigid rules.

For the test, Anthropic planted one anti-pattern in each of six otherwise clean support prompts. It ran them on Opus 4.8, on Opus 5 with only the model ID changed, and on Opus 5 after one audit pass. The audit cut cost by 14.6% and raised accuracy by 5.3% on average. On Opus 5, “verify twice” repeated the order lookup on every refund, and “be maximally thorough” turned into dozens of unneeded knowledge-base searches. The accuracy losses were more concrete. The retired thinking setting made the API reject every routing request. The contradictory refund rules led Opus 5 to withhold four refunds it owed while it asked customers to confirm. And the manual scratchpad clashed with built-in thinking: on three tickets Opus 5 wrote the tool call inside its reasoning and never executed it.

What effort buys

Anthropic’s effort curves are uneven. On FrontierCode Diamond, its 50 hardest tasks, Fable 5 scores 11.5% at low effort for $5.35 per task and 30.9% at max effort for $19.00. That is about 2.7 times the score for about 3.5 times the cost. On Humanity’s Last Exam without tools, Fable 5.1 goes from about 53% at low effort for about $0.30 per question to about 61% at max for about $2.23, so roughly eight points cost more than seven times as much. Most of that comes before the last step: going from the level below up to max adds about half a point for 46% more cost, which Anthropic says is within the benchmark’s run-to-run noise.

Moving to a newer model can matter more than changing effort. On CursorBench 3.2, Fable 5.1 at low effort matches Fable 5 at high effort for a third of the cost. Part of that is pricing: Fable 5.1 cache reads cost $0.25 per million tokens against $1.00 for Fable 5. Even at Fable 5’s prices, Anthropic says Fable 5.1 at low effort would still cost about 40% less.

Hillclimb and cost-optimize

hillclimb splits your evaluation into train and test sets, proposes configuration changes and reads the failing train examples to decide what to fix. Starting from Opus 4.8 at its default high effort, it first tried Opus 5 at low effort with a prompt-audit pass. That configuration beat the baseline at 98.9% train accuracy and 2.6 cents per ticket. Next it tried Sonnet 5 at low effort, at 1 cent per ticket, and accuracy fell to 88.9%. Claude then added routing rules and a refund-cap cross-reference to the prompt, which brought Sonnet 5 back to 98.9% at the same cost. The 90.5% versus 78.6% comparison comes from just 14 held-out tickets that the search never saw.

cost-optimize starts by finding where the money goes. It uses organization usage and cost reports if you have a Claude Admin API key, otherwise the usage object from logged API responses, and as a last resort an estimate based on your request-building code. It then ranks the savings: prompt caching, trimming what each request carries (including a prompt audit), bounding output and batching unattended work. With an evaluation supplied, it also compares effort levels and models. Anthropic ran it on four public benchmarks with Sonnet 5 as the baseline:

• LegalBench: about 58% lower cost from a shared cached prefix, low effort and the Batch API. Thinking tokens fell from 102,779 to 8,284, about a twelfth, and the pass rate stayed within noise.
• tau2-bench retail: 73% lower cost from prompt caching with explicit breakpoints, with a flat pass rate.
• OfficeQA Pro: batching and document caching cut cost from $136.20 to $64.87, about 52%.
• SWE-bench Verified: about 55% lower cost.

On SWE-bench the default config already cached correctly, so the cache was not the source of savings. They came from medium effort and from limiting the agent’s output to a few concise sentences. Median steps per task fell from 29 to 17, and prompt tokens dropped from 75.2M to 33.7M.