Opus 5.5 finds new bugs for CodeRabbit and misses old ones

In CodeRabbit's review of Opus 5.5, the lower-effort Standard setup caught 11 open-source bugs that the production reviewer missed. It also missed nine bugs that the production reviewer caught, so a better score still changes which bugs reach your main branch.

At a glance

• CodeRabbit ran two Opus 5.5 pipeline configurations, Standard and Max, against its production model mix on 80 known open-source bug patterns and 13 harder cases in a benchmark called Signal.
• On the open-source set, Standard caught 51 of 80 issues against the baseline's 49, while Max caught 10 of 13 Signal issues through actionable comments, against five for the baseline.
• Per-token prices fell 20% from Opus 5, yet every Opus 5.5 configuration reported more token usage than the production mix, and the run summary does not convert that into dollars.

If you haven't been following, CodeRabbit says it has benchmarked every Opus release since version 4. According to CodeRabbit, the x-high configuration in its Opus 5 review wrote more precise actionable comments than the baseline (39.3% versus 35.2%) but caught fewer known issues (55.2% versus 61.1%). At the time, CodeRabbit concluded that Opus 5 was not the best general-purpose code reviewer it had tested.

Standard caught 51 of 80 open-source bugs, two more than the production mix

The main test was CodeRabbit's OSS August benchmark: 80 known bug patterns shared by all three configurations. Standard and Max are CodeRabbit's names for setups that span the whole pipeline. Standard combines lower reasoning-effort settings and Max combines higher ones. Neither is a single API effort value or Anthropic's default configuration.

The production baseline caught 49 of 80 (61.3%) with 39.3% actionable precision and 116 reported comments. Standard caught 51 (63.8%) at 38.6% precision with 127 comments. Max caught 50 (62.5%) at 35.7% precision with 140 comments. Max also labeled a larger share of its comments as minor, and CodeRabbit notes that the label does not make a finding invalid.

Precision here is the share of comments that pass the benchmark judge for the target issue. It does not measure whether a developer accepted them. CodeRabbit treats the comparison with its Opus 5 results as historical context, not a measured improvement, because the test sets and configurations differ.

Both Opus 5.5 setups flagged a Cal.com race condition the baseline missed

One of the new catches came from the Cal.com benchmark. Two overlapping jobs could each read a reminder's retryCount as 0, add 1 in application code and write 1 back. Two increments were attempted, and the counter ended at 1.

Both Opus 5.5 configurations spotted the race and proposed the atomic form, retryCount: { increment: 1 }, which tells the database to add one to whatever value it currently holds. With that fix, job A takes the counter to 1 and job B takes it to 2. The production baseline missed the bug.

If you are evaluating a replacement, CodeRabbit suggests listing the bugs only your current reviewer catches next to those only the new one catches, then weighing each by its impact. It also suggests trying a second reviewer alongside the first. These runs say nothing about the quality, cost or comment volume of running both.

On 13 harder Signal cases, Max caught 10 through actionable comments

Signal is a smaller set of harder cases. Through regular actionable comments, Max caught 10 of 13 (76.9%) at 52.0% precision. Standard caught eight (61.5%) at 66.7% precision, and the baseline caught five (38.5%) at 29.4% precision. The three left 25, 21 and 17 reported comments respectively.

Max's 25 comments split into 13 major and 12 minor. Standard's split into 14 and 7, and the baseline's into 11 and 6. The model assigns these severity labels itself, so they are not counts of distinct bugs.

Once findings outside the changed lines were counted, both Opus configurations reached 10 of 13 and the baseline reached seven. Standard and Max missed different issues. That is why CodeRabbit says higher effort did not consistently find more bugs.

Opus 5.5 tokens cost 20% less than Opus 5's, but Standard used 49.2% more than the production mix on OSS August

Input pricing falls from $5 to $4 per 1 million tokens and output from $25 to $20, a 20% cut from Opus 5. Cache reads drop 60%, from $0.50 to $0.20 per 1 million tokens. According to Anthropic, cache reads make up most of the cost of agentic and coding work.

Every tested configuration also reported more token usage than the production model mix. Standard used 49.2% more on OSS August and 40.6% more on Signal. Max used 57.6% more on OSS August and 60.1% more on Signal. These figures come from the team's run summary and do not separate input from output tokens or give dollar cost or latency.

What you pay for a review depends on the per-token price, the number of tokens and the number of calls. Cheaper fuel does not help much if the route gets longer. Thinking also uses up the token budget, so a short visible comment can still come from a large call.

Opus 5.5 no longer lets a pipeline force a tool call

Thinking is now always adaptive: Opus 5.5 rejects requests that explicitly turn it on or off. That makes effort the main control over deliberation, latency and cost. The recommendation is to start at medium and test low and high alongside it. At the same effort setting, 5.5 can think more per turn than Opus 5, especially at x-high and max.

Forced tool calls are retired, so an application cannot require the next response to call a particular tool. Schema enforcement still constrains a call's arguments but does not guarantee the call will happen. It works like a form with strict fields, which controls what gets written but not whether anyone fills it in. Pipelines that depend on a tool result now have to check that the call happened.

Fast mode is available, and the model does not require data retention. Cybersecurity safeguards remain, and a biology classifier is new compared with Opus 5. CodeRabbit says its benchmarks do not measure the effect of any of these changes.

The numbers leave the cost question open. The token increases compare Opus 5.5 with CodeRabbit's production mix, while the price cut compares it with Opus 5, and nothing here turns either into a dollar cost per review. In our view, a 20% discount per token says little about the bill when the same runs report 40.6% to 60.1% more tokens, plus up to 24 more comments per 80 bugs for a developer to read.

What to meter in your own trial

CodeRabbit gives no dollar figure per review and no latency numbers, so your own trial is the next checkpoint. It suggests tracking input, output, cache-read and cache-write tokens separately across a completed review, including retries, verification calls and fallback models. Then weigh the bill against elapsed time, useful findings, lost catches and comment volume. Standard is the suggested starting point, since it gave the better overall balance in these runs.