Skip to content

Benchmarks

Qwen4-Max scores 43 on Artificial Analysis, 9 above Qwen3.8

4 min read

Our AI analyst checked this story itself: it ran, measured and compared against the originals. What we checked

Written by an AI model from the linked sources and checked against them by a second model. How we work

Image: threads.com

Alibaba’s Qwen4-Max scores 43 on the Artificial Analysis Intelligence Index at xhigh reasoning effort. That puts it 9 points above Qwen3.8 Max at the same setting, in results that went up as Alibaba listed the full model on QwenCloud on November 16. The newer model also costs much less to run on this test: Artificial Analysis paid $2,394.31 to put Qwen4-Max through the whole index, against $6,353.83 for Qwen3.8 Max (0902), both at xhigh.

Qwen4-Max lists at $1.50 per million input tokens and $4.50 per million output tokens. Qwen3.8 Max costs $2 and $6. Cached input, meaning prompt text the provider has already processed, drops from $0.25 to $0.075 per million. The model is hosted only, through the QwenCloud API, Token Plan, Qwen Studio, QwenWork and Qoder, and there are no weights to download. It arrived after a preview on November 11, and Alibaba says smaller Qwen4 models with open weights will soon be released.

Alibaba’s own numbers

Alibaba’s launch announcement describes a 2.4T-parameter model that runs only 71B of those parameters for each token. That is 25% fewer than Qwen3.8-Max, which has the same total size. Qwen4-Max reads images natively. Alibaba puts its context window at 1M tokens, and Artificial Analysis lists 984k. On Work360, Alibaba’s in-house benchmark, Qwen4-Max scores 52.69 against 47.74 for Qwen3.8-Max. It uses about 24% fewer tokens at Xhigh effort and 35% fewer at High, and its average cost per task falls 62%, from $4.43 to $1.67. The press release rounds that to around 60%.

Where it trails

Kimi K3.1 Pro is 3 points higher at 46, scored at max effort. Artificial Analysis lists it as open weights, 2.8T parameters with 104B active, so anyone can download it and run it on their own servers. Claude Opus 5.5 (max) is at 59.11. Among the 29 models compared on Qwen4-Max’s page, the top score belongs to Claude Fable 5.2 (max) at 64.56.

Against Kimi K3.1 Pro, Qwen4-Max wins on HLE-Diamond (0.449 vs 0.373). It loses on Terminal-Bench 4.0 (0.485 vs 0.672), Terminal-Bench-Science (0.233 vs 0.343), CritPt (0.43 vs 0.495) and GDP.pdf (0.262 vs 0.314). On AA-Briefcase their 95% intervals overlap, so neither model is ahead there. Compared with Qwen3.8 Max, AA-Omniscience rose from 10.5 to 27.7. The gain came from more correct answers, with accuracy up from 0.295 to 0.437. The hallucination rate did not fall: it was 0.271 and is now 0.284.

Alibaba’s launch chart covers 16 benchmarks. As MarkTechPost transcribed it, Qwen4-Max ranks first on 4 of them and Claude Opus 5.5 leads 7. The chart shows the same weak spots in coding and vision: ProgramBench 31.5 against 65.0 for Opus 5.5, and Chartography 54.2 against 71.1 for GPT-6 Astra. Alibaba says Qwen4-Max ranked 6th on Code Arena. Testing Catalog reported that it debuted at #14 overall on Agent Arena, with a +7.5% net improvement across 14.7k real-world agentic sessions. Alibaba credits its arena scores to the Arena itself.

Our analysis

What we checked

On November 16 we took the scores, token counts and cost figures from Artificial Analysis’s pages for Index v5.0.1. We then split each model’s bill by token type and price.

How we check stories

The bill Artificial Analysis paid fell 62.3%. That matches Alibaba’s Work360 claim in size, but it was measured on a different set of tests, and most of the saving comes from lower prices, not fewer tokens. Testing Catalog’s “one point below Muse Wonder” is 0.68 before rounding, 42.89 against 43.57. Muse Wonder was scored at max effort, while Qwen4-Max has only an xhigh entry.

xhighQwen4-MaxQwen3.8 Max (0902)
Index score42.8933.65
Cost to run the index$2,394.31$6,353.83
Output tokens127M150.5M
  • At Qwen3.8 Max’s prices, Qwen4-Max’s token use would have cost $4,563.96, which is 28.2% less. The new prices account for the rest. For both models, about 92% of input tokens were read from cache, which makes the cached-input cut the biggest factor. This is our own split at list prices; Artificial Analysis does not publish it.
  • Qwen4-Max is still wordy. Artificial Analysis calls it “very verbose”: 127M output tokens against a median of 78.6M.
  • The Terminal-Bench 4.0 numbers in Alibaba’s chart match Artificial Analysis’s measurements for all five models in it. By our inference, the 48.5 for Qwen4-Max is Artificial Analysis’s figure, not Alibaba’s own.

No comments yet. Be the first.

Join the conversation

Sign in to leave a comment. It appears after moderation.