Anthropic’s CI buckled after Claude wrote 80% of the code

In six months, the number of CI jobs at Anthropic grew 25x. Several times that load nearly overwhelmed the service that decides which tests each pull request runs. Three quick fixes held it for 70 days, then 29, then less than one. After that the team threw out the single-process design and rebuilt it as a horizontally scaled system.

An Anthropic engineer described the whole episode in a September 14 post on the Claude blog. The load came from several directions. Anthropic engineers now ship 8x as much code per quarter as they did from 2021 to 2025. Claude writes 80% of that code and does a large share of PR review and approval. The number of tests in the codebase grew 10x, and the company added a nominal number of engineers. The 8x and 80% figures match the Anthropic Institute’s June 4 report on recursive self-improvement, which found that more than 80% of code merged into Anthropic’s codebase as of May 2026 was written by Claude. The 25x figure is not simply those multiples combined, because not every test runs on every PR.

Listener, selector, one writer

Test impact analysis means running only the tests a change can plausibly affect, not the full suite. Anthropic’s version is deterministic: it picks tests based on past performance and package relevance. The author points out that vendors sell this kind of tool. Many of the author’s peers still run every test on every change, and the argument against that is that it stops scaling: CI gates get longer, more expensive and less trustworthy. Agents raise the stakes further. A human can tell which red tests have nothing to do with their change. An agent needs a specific set of valid tests to check its own work and iterate.

The service has two parts that must stay in sync. A listener records the results of every CI run. A selector reads that history and decides which tests run on each newly opened PR. The listener kept a running history for every test, and that required a single writer to apply results in order. So the whole thing ran as one process, and the v0 design could not be sharded horizontally.

With several CI jobs finishing every second, the listener started falling behind the PR queue. A lag of 20 minutes could leave tens of thousands of test updates out of what the selector sees. If a bad change gets merged, a test starts failing for everyone, and several people end up running unnecessary investigations. If a dependency starts flaking, the flaky reds block merges. And a test that was just fixed or added doesn’t run until the listener catches up, which risks a regression.

70 days, 29 days, less than a day

The first signs of strain came in October 2025, when the team got paged two days in a row. Patch one was to double the cores running the service. It lasted 70 days, and the team knew at the time it wouldn’t last. Who owned the fix was unclear. “No one wanted to own another piece of infrastructure,” the author writes, and the CI team had bigger fish to fry.

The pages kept coming as listener lag built up. The author then opened a long-running session dedicated to the service in an internal version of Claude Tag, the agent that also serves as first responder for Anthropic’s CI on-call. Whenever the listener fell more than 50,000 jobs behind, Claude pinged the author and picked up the conversation about next steps where it had left off. This went on for months. “Claude often argued for an overhaul, but we usually settled on another patch.”

Patch two came in February, when the growth curve hit the service again. The team realized the listener didn’t need one global writer, only one writer per package, so results for each part of the codebase stayed in order. Claude generated the code that split each package’s state into its own shard with its own worker. It held for 29 days.

In March the process was hitting its memory limit by mid-afternoon on most weekdays. A search for quick fixes found only four bugs. Swapping the memory allocator did nothing. And nobody wanted to memory-profile a singleton that was already under heavy load. That left restarts, which bought less than a day each and made things worse: with daily restarts the service gradually fell further behind. Several times it was more than an hour behind, and a large number of job results never reached the listener. The author is careful about what that means. CI still ran on those PRs and no untested code reached production. But the selector was choosing tests from stale data, which mostly meant running tests that were already very flaky or failing across the board.

What replaced the singleton

The redesign followed Claude’s advice: give the test selection service a database, specifically an in-memory data store. A listener worker can now take any result, append it to a journal in the store and move on without keeping anything in its own memory. Because the workers hold no state, the listener scales horizontally. A small separate consumer process rolls the journal up into per-test history every few seconds, and the selector reads that history quickly.

• Test history: inside the singleton’s memory → in an in-memory data store, built from a journal
• Who writes results: one process, later one worker per package shard → any stateless listener worker

The trade-off is cost. The distributed version is more expensive to run, but it is much easier to scale and to memory-profile than the old singleton. One engineer built it in three weeks, and the author estimates the same project would have taken closer to a quarter a year ago. That is the post’s real point: bigger machines, parallelizing and restarts each buy a fraction of the time they did a year ago, while a full redesign now also takes a fraction of the time. Claude did most of the fine-tuning, sizing the journal and the worker count, largely on its own. A chart in the post tracks the hourly peak of queued, unprocessed job-result events. Before the switch, a backlog built up on most days and grew week over week. After the cutover and tuning, the line is flat, and the service has stayed stable since.

Why the load keeps climbing

The author expects CI jobs to keep growing exponentially as engineers run more agents each and as accelerated PR approval gets more sophisticated. Claude also prefers smaller, more granular PRs, which means more CI jobs per day, and the author calls this another reason not to run every test on every PR. Agents push overnight and on weekends, which raises the baseline activity level. Load is still bursty, because human engineers still drive and approve a significant share of PRs. The author anticipates horizontally scaled test selection becoming an industry standard as agent-heavy teams produce more PRs and more tests.

The advice applies whether a team builds or buys: “assume your architecture will be at a 25x load within two quarters.” If the budget allows, v0 designs can now account for 10-20x the perceived scale, and the author says over-engineering as a concept is starting to fade, or at least the bar has moved much higher. Instrument services so they act as Claude’s eyes and ears, which lets it fix problems incrementally much faster than people could by hand, and check that the number of CI jobs coming in equals the number going out. Avoid running any critical service as a single instance unless you can measure it and any canary changes. And, from the same list: “Keep state out of the process from the start.”