Skip to content

OpenAI

Cache writes ate 85% of a Codex Bedrock bill

3 min read

Written by an AI model from the linked sources and checked against them by a second model. How we work

All about: OpenAI models and products →
Image: github.com

Cache writes made up about 85% of an estimated $1,386.46 that one Codex CLI user spent on GPT-5.6 Sol through Amazon Bedrock between August 5 and August 8, according to a bug report filed on GitHub against the Codex repo on August 21. The reporter traces the cost to one gap: native Codex requests to Bedrock Mantle can’t opt into the model’s explicit prompt caching. The report links that gap to the large volume of cache-write tokens on an agentic coding workload and to the higher cost.

The reporter took usage quantities from Cost Explorer and priced them with the Bedrock rate card, so the figures are estimates and not finalized AWS invoice amounts. Over 3,656 requests, Sol wrote 171.94 million tokens to cache, at an estimated $1,182.09. One local session shows the same pattern on a smaller scale: 76 Sol requests, 6.709 million cache-write input tokens, zero cached input tokens, and about 88K tokens written per request on average. CloudWatch showed no client errors for the same traffic.

The setup is Codex CLI 0.147.0 with the native amazon-bedrock provider, calling the Bedrock Mantle Responses API in us-east-1. Codex already sends a session-scoped prompt_cache_key. But the request types for both HTTP and WebSocket Responses calls include neither prompt_cache_options nor prompt_cache_breakpoint, the marker on the content block where a reusable prefix ends. The built-in Bedrock provider config covers only transport and auth settings, so you can’t add the fields through config.toml. AWS documents explicit cache mode for GPT-5.6 on Bedrock for agentic workflows: long, stable instructions and tool definitions, followed by tool and user content that keeps changing. The reporter says that matches this workload.

An earlier report on GitHub, which the new one builds on, explains why the default behavior falls short. GPT-5.6’s implicit caching places a managed breakpoint near the latest user or tool message. A large stable prefix followed by a changing suffix can therefore miss the cache even when the prefix hasn’t changed. That report ran a controlled replay on Bedrock Mantle, tested as of July 24, with a developer prefix of roughly 9k tokens. Unmodified requests got 0% cache hits and rewrote about 9,060 tokens each. Adding an explicit breakpoint to the last stable input_text block raised hits to 98.6% and cut warm-request writes to 123-128 tokens. The same report found that append-only turns within one session already reached about 98% hits; the failure it demonstrated is in reusing a stable startup prefix across independent requests. It also quotes Codex’s vendored migration guide: “Cache writes cost more than ordinary uncached input, so a lower hit rate can be both slower and more expensive.”

The new report asks for four changes:

  • serialize prompt_cache_options for Responses providers that support GPT-5.6;
  • add a typed prompt_cache_breakpoint field to the supported input content blocks;
  • gate the feature by provider and model, and place the breakpoint safely at the end of Codex’s measured stable instruction and tool prefix;
  • show cache reads and writes in per-turn usage telemetry, so users can spot costly full-prefix rewrites.

Cache parameters have caused trouble for Sol in Codex before. In the next release, CLI 0.148.0 sent prompt_cache_retention to the model, and every turn failed.

The reporter keeps the claim narrow. Cold starts, genuinely distinct prompts, forks and compaction can all require cache writes, and the report does not call every write a defect. The complaint is that “native Bedrock Codex currently has no way to use the documented explicit-cache mechanism for the stable-prefix case.”

No comments yet. Be the first.

Join the conversation

Sign in to leave a comment. It appears after moderation.