CodeRabbit: Opus 5.5 trades 9 missed bugs for 11 new ones

Swap Claude Opus 5.5 in as your code reviewer and the bug count barely moves, but the list of bugs changes. In CodeRabbit's evaluation, the Standard setup caught 11 open-source issues its production reviewer missed and missed nine that reviewer caught, with bigger gains on harder cases and a larger token bill.

At a glance

• CodeRabbit ran two Opus 5.5 setups, Standard and Max, through its review pipeline against its production model mix, on 80 open-source bug patterns and 13 harder cases called Signal.
• On the open-source set Standard caught 51 of 80 known issues against the baseline's 49, while on Signal the high-effort Max setup caught 10 of 13 and the baseline caught five.
• Per-token prices fell 20% relative to Opus 5, yet across both benchmarks every tested setup used 40.6% to 60.1% more tokens than CodeRabbit's production mix and left more comments.

If you missed the previous round: when CodeRabbit evaluated Opus 5, its x-high configuration wrote more precise actionable comments than the production baseline but caught fewer known bugs. Opus 5.5 flips that shape, with slightly more open-source catches and slightly lower precision. CodeRabbit warns that test sets and configurations differ, so this is historical context rather than a measured improvement between versions.

Standard caught 51 of 80 open-source bugs, two more than the production reviewer

Standard and Max are pipeline-wide configurations, one combining lower reasoning-effort settings and one higher, not a single API effort value. On the OSS August benchmark, the baseline caught 49 of 80 (61.3%) at 39.3% actionable precision with 116 reported comments. Standard caught 51 (63.8%) at 38.6% with 127 comments. Max caught 50 (62.5%) at 35.7% with 140, labeling a larger share as minor.

The totals hide a swap: 11 new catches for Standard, nine lost ones. One new catch came from Cal.com, where two overlapping jobs both read a retry counter of 0 and each wrote back 1. Two increments were attempted, and the counter ended at 1, like two cashiers each adding a sale to the same old total.

Both Opus 5.5 setups flagged the race and proposed an atomic increment, so the database adds 1 to whatever value it currently holds and both updates survive. The production baseline missed it.

On 13 harder Signal cases, Max caught 10 and the baseline five

Signal is a separate, smaller benchmark of harder cases. Through regular actionable comments, Max caught 10 of 13 (76.9%) at 52.0% precision, Standard caught eight (61.5%) at 66.7%, and the baseline caught five (38.5%) at 29.4%. Max left 25 comments, Standard 21 and the baseline 17.

Those comments carry model-assigned severity labels, which are not counts of distinct bugs. Max split its 25 into 13 major and 12 minor, Standard its 21 into 14 major and 7 minor, and the baseline its 17 into 11 major and 6 minor.

Counting findings outside the changed lines shifts the result. Both Opus setups then reached 10 of 13 and the baseline seven, but Standard and Max missed different issues. CodeRabbit's reading is that higher effort did not consistently find more bugs.

Prices per token fell 20%, but review runs used 40.6% to 60.1% more tokens

Against Opus 5, input pricing drops from $5 to $4 per million tokens and output from $25 to $20. Cache reads fall 60%, from $0.50 to $0.20 per million. Among the listed changes, Opus 5.5 at medium effort matched or beat Opus 5 at high effort on multi-step coding tasks with roughly half the tokens.

CodeRabbit's own runs measured usage against its production mix, not against Opus 5. Standard used 49.2% more tokens on OSS August and 40.6% more on Signal; Max used 57.6% and 60.1% more. The run summary does not separate input from output tokens or establish dollar cost or latency.

A short visible answer can still use substantial tokens, CodeRabbit notes, because thinking draws on the same budget. Its advice is to ask for concise findings, then track input, output, cache-read and cache-write usage across the whole review, including retries, verification calls and fallback models.

Outside review, Opus 5.5 built a San Andreas-style game and bots for three of them

Gowtham, who worked on the evaluations, said on CodeRabbit's podcast that he gave Opus 5.5 a goal for a personal coding project and let it work overnight, with results that impressed him and the team. CodeRabbit also compared three games inspired by Grand Theft Auto: San Andreas: Opus 5.5's Palmera Bay, Astra's Westline and Fable 5.1's Sunhaven.

Palmera Bay stood out for its detailed environment and range of features, though it took longer to build than the Fable example. Opus 5.5 then wrote bots that read live game state and send keyboard and mouse inputs: stunt driving and combat in Palmera Bay, a coastal mission in Westline, a street race and police escape in Sunhaven.

Effort replaces the thinking switch, and no tool call is guaranteed

Thinking in Opus 5.5 is always adaptive: the model rejects requests that explicitly turn it on or off, so effort becomes the main dial for deliberation, latency and cost. The advice is to start at medium and test low and high alongside it. At the same setting, 5.5 can think more per turn than Opus 5, especially at x-high and max.

Forced tool calls are retired, so an application cannot require the next response to call a particular tool. Schema enforcement still constrains the arguments of a call, but does not guarantee the call itself. Workflows that depend on a tool result must check that the call happened and handle the case where it did not.

Deployment options shift as well. Fast mode is available, the model does not require data retention, cybersecurity safeguards remain and a biology classifier is new relative to Opus 5. CodeRabbit says its benchmarks do not measure the impact of any of these.

The numbers carry limits CodeRabbit states itself. Precision here is the share of comments passing a benchmark judge, not developer acceptance, and the token figures stop short of a cost. In our view, the 20% price cut is the less useful number for a review team, since it is measured against Opus 5 while the 40.6% to 60.1% token increases are measured against the production mix, and the two cannot be netted into a bill.

The dual-reviewer bill, unmeasured

CodeRabbit gives no dollar cost or latency for any configuration, and says its runs do not establish the quality, cost or comment volume of running Opus 5.5 as a second reviewer next to the current one, the setup its differing catches point toward. Those numbers remain for each team's own trial, with Standard as the suggested starting point and Max tested for the important bugs it adds or loses.