CodeRabbit: Opus 5.5 trades 9 missed bugs for 11 new ones

Swap in Opus 5.5 as your code reviewer and a different set of bugs slips through. In CodeRabbit’s open-source benchmark, the Standard setup caught 11 issues its production reviewer missed and missed nine that reviewer caught. The net result was slightly better coverage with slightly lower precision.

On 13 harder bugs in a benchmark called Signal, the high-effort Max setup caught 10, and the baseline caught five. Both Opus setups spotted a Cal.com race in which two jobs overwrote each other’s retry counter, and both proposed an atomic increment.

The catch is the workload. CodeRabbit says Standard used 49.2% more tokens than its production mix on the open-source test and left more comments to read. The price cut is measured against Opus 5, a different comparison.

Cheaper tokens, just a lot more of them.