Opus 5.5 beats GPT-6 Astra on data-analysis repeatability

Claude Opus 5.5 leads Deepsense’s latest EDA Benchmark leaderboard with a reliability-adjusted score of 0.62. GPT-6 Astra is next at 0.55, and their mean scores are close at 0.65 and 0.63. The post went up on November 2 under an October 28 title. If your team is choosing a model for data-analysis agents, our recomputation from the published runs supports the top two places. Most of the gap comes from a single task where Astra swung between perfect and near-zero runs, and several places further down depend on how the scoring code treats tasks a model failed completely.

The benchmark gives each model 10 synthetic data-analysis tasks: inspecting files, cleaning inconsistent records, combining sources, detecting anomalies and returning structured results. Each model runs every task five times. The mean score measures average quality. The reliability-adjusted score lowers that mean when a model’s runs on the same task disagree with each other. Models from the July round were not re-tested and stay on the board as a reference. GPT-5.6 Sol keeps its 0.46 and now shares third and fourth place with GPT-6.1 Sol, which has a mean of 0.59. GPT-6 Sol follows at 0.43 with a mean of 0.53. Gemini 3.8 Flash and Claude Fable 5 both sit at 0.42.

Where the Opus lead comes from

Deepsense’s only explanation is that the gap between Opus 5.5 and Astra “widens after adjustment” because Astra is less stable. We traced that to one task, store-audit, where Astra scored 1.00 three times and 0.063 three times. Its adjusted score on that task is 0.06, while Opus 5.5 has 1.00 from five perfect runs. Astra also has a lower mean than Opus on solar-farm-daily-yield, 0.263 against 0.585. The lead does not cover every task. On hospital, Astra scored 1.00 in all five runs and Opus 5.5 scored 0.50 each time. On identify-unusual-values, Opus 5.5 averaged 0.024, while Claude Fable 5 scored 0.958 in all five runs in the July round.

The order still held when we resampled the five runs within each task 10,000 times: Opus 5.5 finished ahead of Astra in 98.5% of the draws, with the gap ranging from +0.04 to +0.145. This resampling covers run-to-run variation only. It does not resample the tasks themselves, and there are only 10. A narrower result appeared in October on Artificial Analysis’ Intelligence Index v5, where Opus 5.5 scored 58 and GPT-6 Astra 57, both at their max settings.

GPT-6 Sol against GPT-5.6 Sol

Deepsense presents this pair as its clearest example. GPT-6 Sol has the higher mean, 0.53 against 0.48, but finishes behind after adjustment, 0.43 against 0.46. The post concludes that GPT-5.6 Sol “is the more dependable choice.” In our resampling, though, GPT-6 Sol came out ahead in 26% of the draws, and the interval for the gap runs from -0.063 to +0.054, so it includes zero.

One scoring rule also affects this pair. When a model scores 0.0 in all five runs of a task, the code drops that task from the adjusted average, but the task still counts as zero in the mean. This happens in nine model-and-task combinations, including GPT-5.6 Sol and Gemini 3.8 Flash on hospital. With those tasks counted as zero, the order changes:

Opus 5.5, Astra, GPT-6.1 Sol and GPT-6 Sol have no task where every run scored zero, so the rule doesn’t change their scores.