Mistral Large 4 tops open models at just 15% on legal tasks

A 15% score sounds like a failure. Per VentureBeat, though, it’s the best open-weight result on Harvey’s Legal Agent Benchmark, and it belongs to Mistral Large 4, nicknamed Le Chonk, a 1-trillion-parameter model that outputs text only.
VentureBeat’s report puts the same model at 62% on DeepSWE, ahead of GLM-5.3. It also scores 67% on Finch, a set of financial tasks, which is again the top open-weight result. TNW reports that the weights are planned for October 27. When Mistral’s CEO spoke to Reuters on announcement day, the remarks named no model and gave no scores.
The catch is that these numbers come from press coverage, and there’s no technical report showing how they were measured. Would you trust them before the weights are out?
Up next
Claude Startups widens its door, with up to $45K in perks, Video
A founder with a freshly funded company can apply to Claude Startups for a free year of Claude Team and $1,000 in API credits on approval. Per CNBC, discounts on the rest of the startup’s tools add up to $45,000.
Related stories
Comments
No comments yet. Be the first.
Join the conversation
Sign in to leave a comment. It appears after moderation.

