A 125B MoE model previews Qwen4’s architecture

On August 26, Alibaba put its new 125B-parameter Qwen3.8 Flash on the QwenCloud API at $0.16 per million input tokens and $0.47 per million output tokens. The company also says Flash runs on a new architecture that leads into the one Qwen4 will use. The details are in a post on Threads.
Flash is a multimodal Mixture-of-Experts model, so each token passes through only part of its 125B parameters. It has a native context of 262K tokens, and YaRN stretches that to 1M. YaRN rescales the model’s position encoding so it can handle sequences longer than the ones it was trained on. The window matches the dense Qwen3.8-27B that Alibaba opened on August 14 and the Max-class Qwen3.8-2.4T-A95B.
On coding, Flash scores 58.7 on DeepSWE 1.1 and 62.5 on SWE-bench Pro. Alibaba’s August figures for the dense 27B model put it at 42.2 on DeepSWE, up from 14.2 for Qwen3.7-Plus. Both sets of numbers come from Alibaba.
The Qwen3.8 repository says the rest of the 3.8 family is built on the Qwen3.5 foundation. That foundation pairs Gated Delta Networks with sparse Mixture-of-Experts for high-throughput inference. Gated Delta Networks are a form of linear attention, so their cost grows in step with sequence length instead of with its square. The 2.4T-A95B flagship, which reached Hugging Face on August 12, combines Gated DeltaNet with gated full attention and sends each token to 11 of its 512 experts. Of the 3.8 models, Flash is the one Alibaba ties to Qwen4.
The weights are on Hugging Face as Qwen3.8-Flash-Next.
Up next
Stealth OpenRouter model Ox Alpha linked to GLM-5.3
Developers who ran the stealth Ox Alpha model on OpenRouter now have a name for it, per a Threads post: Zhipu’s GLM-5.3-Flash, 320B-A18B under MIT, though Kingy AI says the Flash label stays unverified.
Related stories
- Qwen3.8-Omni-Flash takes video and audio in a 1M window
- Open weights land as a dry run for Qwen4
- 27B open weights ship under Apache 2.0
- Mistral Large 4 weights are out, API now from $0.70 per 1M
- Kimi K3.1 Pro lands just behind Opus 5.5 on Terminal-Bench
- DeepSeek’s minor Flash update beats Opus 5 on DeepSWE
Comments
No comments yet. Be the first.
Join the conversation
Sign in to leave a comment. It appears after moderation.

