FutureOS kept 147 of 178 answers, Codex kept 68

Ask an agent what happened three hours ago and, nine times out of ten, you are asking about words — yours or its own — not about tool output, which is 96.1% of everything in the window. A compaction benchmark from the FutureOS team, published on Github, prices that mismatch: 147 of 178 answers retained after a forced compaction, against OpenCode's 83 and Codex's 68.

At a glance

• The test grows one session until the window is full, forces a compaction, then asks 178 questions answerable only from memory; eight are decoys, and no system guessed at one.
• Codex keeps user messages plus a whole-history summary and drops assistant prose outright, scoring 68 of 178; OpenCode's summary plus a 15,000-token tail scored 83; FutureOS's originals-first projection, 147.
• Both rival columns are single-point reimplementations pinned to Codex b13164d8 and OpenCode package 1.18.31, the exam tests exact-value recall rather than task continuation, and the cache prices are modelled.

If you have not been following: a serious refactor — reading files, running tests, writing patches, re-reading them — can saturate a 200k-token window inside one working session, which is why every coding agent ships a compaction path, according to the Codex Knowledge Base. OpenCode's documentation says its compaction is on by default, keeps about 15,000 tokens of recent conversation beside the generated summary, and warns plainly that compaction is lossy.

Tool output is 96.1% of the characters and 10% of the questions

The team tallied five frozen real-session chains: 9,665 records, 6,889,611 characters. Tool output accounted for 4,370 of those records and 96.1% of the characters. Assistant text was 784 records and 3.7%; user text, 141 records and 0.2%. Per chain, tool output never fell below 93.2% of the characters.

Then they looked at where the later "what happened earlier" questions point. 80% of them ask about something the assistant itself said, 10% about user turns, 10% about tool output. Everything you and the model said to each other is 3.9% of the bulk and 90% of the references.

Codex keeps no assistant text at all

Each system answers the same question differently. Codex keeps all user messages, capped at 20,000 tokens, plus a whole-history summary; assistant prose and tool output are dropped. OpenCode keeps a summary plus a recent tail capped at 15,000 tokens, and expects detail to survive inside the summary. FutureOS keeps protected user and assistant originals first, with the summary and a tool-evidence index as compensation for what does not fit.

According to the Codex Knowledge Base, the reconstructed context after a Codex compaction holds exactly two things — one summary message in the user role plus a portion of recent tokens — and on non-OpenAI providers Codex asks the model for a handoff summary and stores it as a user-role message prefixed with _summary. Codex's first-party retrieval tools need its hosted backend, so the column measured here is that local fallback.

What actually happens at 800,000 tokens?

The keyword is projection. Originals are never deleted or rewritten; compaction recomputes, for the next request only, what the model gets to see, while the journal stays on disk — searchable, exportable, forkable. The trigger runs before every model step: floor(W × 0.8), so 800,000 on a 1M window, bounded by W minus the model's declared output ceiling minus a margin of min(2048, W/16). If the input still fits, nothing is cut.

Inside the projection, user text is always kept, because user constraints and goals cannot be regenerated while prose and tool output usually can. The remaining room prefers assistant originals. Tool output is compressed into a 2,048-token evidence index, a recent tail of about 8K tokens survives, and the history target is roughly 32K.

The index takes no model call. Errors come first, then groups by tool and target with config, schema, validation and test targets prioritised; each row is bounded JSON, never cut in half, priority-ordered rather than chronological, with sourceOrder kept so an old error cannot pass for a current one. Closer to a back-of-book index than to a retelling of the chapters.

83% costs ¥0.53 cached, 38% costs ¥0.42

The default, summarized-evidence-v1, projects 12,113 tokens where an uncompacted session sends 232,777; the harness scores that 5.7% compression. OpenCode's projection is 5,152 tokens, Codex's 1,706. Codex is the cheapest to run: ¥0.42 per cached compaction, ¥0.000068 per later turn, break-even at 46 turns against 61 for the summarized default and 110 for OpenCode.

FutureOS's summary request reuses the session's own system prompt and tool definitions, so the provider's prefix cache serves it and ¥7.98 cold becomes ¥0.53. On a primed prefix, the identical shape hit 93.7%; substituting the system prompt, dropping the tool definitions, or adding one line each measured 0%. On the real path, a session grown to 212,911 tokens compacted with cache_read 212,548 — 99.8%, ¥0.003.

Over one compaction plus 100 turns, a point of recall costs ¥0.0070, ¥0.0112 and ¥0.0217 for FutureOS, Codex and OpenCode. There is a second tier as well: deterministic-evidence-v1 makes no model call, projects 9,945 tokens, answers 127 of 178, and doubles as the fallback when no provider is reachable.

Given a search tool, the model never used it

If the originals are all still there, why not let the model look them up? They ran the open-book version on the summarized arm — 18 cases, 178 values. The closed baseline answered 147. Production prompt plus tools, autonomous: 148. Stronger recall guidance: 149. Wording that stops calling the projection "the record": 145. Tool calls across all three rounds: zero.

A control re-scored the closed arm under the third round's wording and got 147 either way, so the spread is one-flip noise. Twenty-seven values stayed unanswered in every round where searching was merely allowed. Forcing the first call, with an explicit verification instruction, produced 166 of 178 (93.3%) across 100 tool calls and recovered 20 of the 28 absent values — an upper bound, since no such instruction ships in production.

So the recall guidance came out of the runtime and the retrieval CLI stayed. Three variants, 54 cases, zero changed behaviour. A side effect: a session's system prompt stays the same when a checkpoint is committed.

The exam asks for exact values — recognition, which verbatim retention is best at and a summary worst at — on one draw per cell, with tool-heavy synthetic fixtures and cache prices modelled against a single production measurement. In our view the striking row is the free one: the deterministic tier needs no provider and no model call, and still lands within 12 points of the paid default.

Before you quote a number

The harness ships a check, verify_request_shape.py, that needs no model and confirms the exam is still sending production's prompt and tools. The comparison columns expire on their own: a later upstream commit to Codex or OpenCode invalidates them. And the real-session chains cannot be published at all, so reproducing that half means substituting your own sessions — absolute numbers will differ, while the comparisons should hold.