Claude went from bug hunt to abuse report on New Year's Eve

On New Year's Eve, a spike of HTTP 500 errors from Opus led Claude Code to a bug that hit only requests carrying exactly 22 images, and from there to 200 accounts in a batch of 4,000 that turned out to be abuse. Alex Palcuie, who handles incidents for Claude at Anthropic, told that story at Signals Berlin. It was part of his case for moving his answer to "Can Claude fix itself?" from "no" in March to "maybe".

The account comes from a write-up on the Sylvainkalache blog. Its author works at Rootly, which builds AI-assisted incident response tools. It covers three incidents Palcuie described. In two, Claude caught something humans missed. In the third, it chased a convincing wrong explanation.

At a glance

• Palcuie described three production incidents at Signals Berlin: an abuse campaign hidden behind an image bug, a monitoring glitch that fooled a router, and a cache red herring.
• He grades the model with the Air Force's OODA loop: Claude is strong at observing and deciding, jagged at orienting, and acting in production still needs a human to approve.
• The write-up argues the bigger limit sits outside the model: an agent is only as good as its context, and many teams lack the logs, access or budget Palcuie has.

According to InfoQ, Palcuie worked as an SRE on Google Cloud's compute product before Anthropic and sat on the "SRE for SRE" escalation team. The write-up adds that he rose to Staff level during eight years at Google. InfoQ also reports that he opened an earlier QCon London talk with "It's a no," and quotes him saying that "since about January this year" he reaches for Claude before his monitoring dashboards.

Twenty-two images led Claude from a preprocessing bug to 4,000 suspicious accounts

On New Year's Eve the errors looked like a plain engineering problem. Opus was returning more HTTP 500s than usual. Palcuie asked Claude Code to investigate, and it traced the failures to a bug in image preprocessing: every failing request carried exactly 22 images. Palcuie says that at this point he would probably have filed a ticket for the API team and left it there.

Claude kept going and checked who was sending those requests. It found 200 accounts sending similar requests at the same time, all from a batch of 4,000 created three days earlier with the same email pattern. It recommended bringing in the safeguards team, and they confirmed the abuse. Palcuie calls this his Move 37 moment, after the AlphaGo move against Lee Sedol that experts first took for a mistake and that helped the AI win.

A few servers reported more tokens per second than was physically possible

In the second incident, the inference router drained a healthy cluster and overloaded the servers that took on its traffic. The humans in the channel blamed the data center. Claude noticed that a handful of servers were reporting more tokens per second than they could physically produce. It concluded that the system was fine and the measurement was wrong.

It traced the bad numbers to the monitoring system. The metric collectors had just been resharded, and for five minutes two shards scraped the same servers. Picture two cashiers scanning the same shopping cart: the sales figure doubles, but nothing extra was sold. The inflated rate was the number the router relied on. Claude proposed and tested a fix while the team paused the collectors' autoscaler to prevent a repeat.

Palcuie counted this as a second Move 37, because Claude had gone past his own infrastructure and into the observability stack, a system Palcuie himself is not an expert in.

A 41% error spike that Claude pinned on the wrong cache

The third story went the other way. One cluster returned errors for 41% of requests over five minutes. Claude noticed that local cache hit rates had dropped just before. A lost cache explained the errors neatly, and the timing made the story convincing.

An hour later, another engineer's Claude session helped uncover the missing piece. Requests that missed the local cache still pulled data from a shared cache. The overall hit rate had barely dipped, and the real problem was the overloaded servers. Palcuie says he could have made the same mistake, because the graph fit the explanation he expected.

He recalled a warning from Steve McGhee, his first manager at Google: looking at graphs without an explicit reason can turn up problems that are not really problems. An agent can search far more graphs than a person can, and that also gives it more chances to find a misleading pattern.

Most teams start further back than Anthropic, the Rootly author argues

The write-up's author points out that Palcuie has access to Anthropic's best models and almost unlimited tokens. He argues that most teams start further back, for reasons that have nothing to do with the model. Code and infrastructure may lack instrumentation, security restrictions may block access to data that exists, and key context may live only in people's heads.

Cost is the other wall. The author says engineers have repeatedly told him that sending application logs to Datadog cost far more than their budget allowed. That left their incident response agent useless, because it lacked the context it needed. In his view these tools can reduce toil for on-call engineers, but a fully autonomous setup is still far away.

Palcuie scores Claude on the four steps of the OODA loop

To judge how far self-repair goes, Palcuie borrows the OODA loop from the United States Air Force: observe, orient, decide, act. The idea is that whoever makes sense of a changing situation fastest gains the advantage. He argues that this holds in an incident channel as much as in a cockpit.

Observing is where LLMs shine. They search logs, query metrics and follow several paths in parallel, faster than any human responder. Orienting, where they have to make sense of what they found, is where results get jagged. Deciding is in good shape: Claude can propose fixes, compare their trade-offs and test them against past incidents.

Acting is where the line is still being drawn. Palcuie wants human approval for any production fix, and he wants to understand every command he runs. For more autonomy, he argues, you would need a production action classifier that approves routine operations and flags riskier ones for review. A bank works the same way when it lets a coffee purchase through but calls you about a large transfer.

All three incidents are anecdotes from one engineer at the company that makes the model, and the write-up gives no rate for how often Claude's first theory holds up. In our view, the detail worth copying is how the cache mistake got caught. The wrong theory survived about an hour and fell only when a second engineer's separate Claude session looked at the problem, which suggests running parallel investigations rather than trusting one confident one.

Before Claude gets the approve button

Palcuie's condition for more autonomy is a production action classifier. The write-up names no plan, team or date for building one, so for now every production fix still waits for a human. The Rootly author expects humans to stay in the loop for quite some time. For most teams, the question to settle first is whether their agent can even see the logs and metrics it would need.