Mistral Large 4 tops open models at just 15% on legal tasks
A 15% score sounds like a failure. Per VentureBeat, though, it's the best open-weight result on Harvey's Legal Agent Benchmark, and it belongs to Mistral Large 4, nicknamed Le Chonk, a 1-trillion-parameter model that outputs text only.
VentureBeat's report puts the same model at 62% on DeepSWE, ahead of GLM-5.3. It also scores 67% on Finch, a set of financial tasks, which is again the top open-weight result. TNW reports that the weights are planned for October 27. When Mistral's CEO spoke to Reuters on announcement day, the remarks named no model and gave no scores.
The catch is that these numbers come from press coverage, and there's no technical report showing how they were measured. Would you trust them before the weights are out?
