A 125B MoE model previews Qwen4's architecture

Alibaba has released Qwen3.8 Flash, a multimodal Mixture-of-Experts model with 125B parameters and a native context window of 262K tokens. The release was posted on Threads, with the weights published on Hugging Face as Qwen3.8-Flash-Next and hosted access sold through the QwenCloud API.

At a glance

• The Mixture-of-Experts design sends each token through only part of the 125B parameters, and the model sits on a new architecture that serves as a precursor to the one Qwen4 will use.
• On coding benchmarks it scores 58.7 on DeepSWE 1.1 and 62.5 on SWE-bench Pro, while QwenCloud prices it at $0.16 per million input tokens and $0.47 per million output tokens.
• The context window runs to 262K tokens natively and extends to 1M through YaRN, matching the 27B dense model Alibaba opened earlier this month, with weights available on Hugging Face.

The pricing and the benchmark selection read as a bid for coding agents that run long sessions over large codebases, where token volume rather than peak reasoning quality sets the bill. The architecture note carries further than the price. If Qwen4 inherits the design, Qwen3.8 Flash is the first public read on how Alibaba's next generation behaves, and its two coding scores become a baseline for that generation rather than a verdict on this one.

Qwen3.8 Flash scores 58.7 on DeepSWE 1.1 and 62.5 on SWE-bench Pro

Qwen3.8 Flash scores 58.7 on DeepSWE 1.1 and 62.5 on SWE-bench Pro, the two software engineering benchmarks published with the release. Hosted access runs through the QwenCloud API, which charges $0.16 per million input tokens and $0.47 per million output tokens for the model.

Alibaba has also published the weights on Hugging Face as Qwen3.8-Flash-Next, so the scores can be rechecked outside the vendor's API. SWE-bench Pro belongs to the SWE-bench family of evaluations, which score a model on how many real repository issues it can resolve end to end rather than on isolated code completion. DeepSWE 1.1 is the second suite Alibaba reported against, and it returned the lower of the two figures.

YaRN takes the 262K native window to 1M tokens

The native context window holds 262K tokens. Alibaba extends it to 1M with YaRN, an interpolation method that rescales rotary position embeddings so a model can process sequences far longer than those it saw during training. The 1M figure is therefore a configuration option rather than a length the model was trained on from the start.

The window matches the 27B dense model Alibaba opened earlier this month, which carries the same 262K native span and the same 1M ceiling under YaRN. What separates the two releases is scale and routing: 125B parameters spread across experts against 27B activated in full on every token.

The 125B parameters run on an architecture that precedes Qwen4

Qwen3.8 Flash is a Mixture-of-Experts model, so its 125B parameters are divided among experts and each token passes through only a subset of them. That structure keeps the compute per token, and with it the serving cost, below what a dense model holding the same total parameter count would require. The model is multimodal.

The release presents the model as built on a new architecture that serves as a precursor to the one Qwen4 will use. Alibaba gives no breakdown of what changed against the current Qwen3 generation, and the parameter count with the routing scheme is the only structural detail attached to Qwen3.8-Flash-Next.

What Qwen4 inherits

The release presents Qwen3.8 Flash as a precursor to Qwen4's architecture but gives no timing for that model. Alibaba published no scores beyond the two coding benchmarks and no comparison figures for competing models, so reasoning, math and multimodal results are unavailable. Whether the DeepSWE 1.1 and SWE-bench Pro figures hold up in independent runs is still open.