Open weights land as a dry run for Qwen4

Alibaba has opened the weights of Qwen3.8-Flash-Next, a 125-billion-parameter multimodal Mixture-of-Experts model it presents as an early preview of the architecture Qwen4 will use, The New Stack reported on August 28. If your team plans to self-host, Flash-Next on Hugging Face and ModelScope is the download; the 1M-token default context and official built-in tools come with Qwen3.8-Flash, the production version on the QwenCloud API, priced at $0.16 per million input tokens and $0.47 per million output.
Which name is which
Qwen3.8-Flash-Next pairs a 125B-parameter main model with another 51B parameters of N-gram embeddings and activates 6B parameters per token. It handles 262,144 tokens of context natively and 1,000,000 with YaRN.
The hosted Qwen3.8-Flash is also reachable through Model Studio, and developers inside China pay 3 RMB per million output tokens. On QwenWork, Alibaba’s workplace agent platform, it runs a redesigned “standard mode” that cuts token consumption per task by 75% and “roughly doubles generation speed” compared with the current mode.
62.5 on SWE-bench Pro
On SWE-bench Pro, an agentic coding benchmark, Qwen3.8-Flash-Next scores 62.5, against 56.0 for DeepSeek-V4-Flash-0731, 55 for Qwen3.7-Plus and 53.4 for Claude-Opus-4.6 (Max). The dense Qwen3.8-27B from the same family sits only 0.8 points behind at 61.7, and a 4-bit Mac build of that model fit in 16.1GB in August.
Qwen3.8-Flash needed “around one-ninth of the training resources” of Qwen3.7-Plus, a model three times its size, Alibaba says, and “significantly reduces” both training and inference costs against it.
Running it locally
Developers can run the model on hardware they likely already own, Alibaba claims. Alok, a mechatronics engineer, posted on X that he ran Qwen3.8-Flash-Next with a 250,000-token context window, inside its native range, on a single 24GB RTX 4090, getting 21 tokens per second decode and 364 tokens per second prefill. He used no MTP, no dflash and no KV cache quantization. “We are running datacenter models on consumer hardware,” he wrote.
Embedding Shapes, a developer posting on Hacker News, is less happy: “I too got hopeful reading the sentiment on the Internet about Qwen 3.8, but it’s just not there (yet) for local professional software development.”
The previous architecture preview hit trouble in local tooling: when Qwen3-Next arrived on September 10, 2025, heise online reported that its significantly changed architecture caused problems with llama.cpp, which would probably not work with it for the time being, while Transformers, vLLM and Apple’s MLX did.
Inside the four upgrades
Alibaba says Flash-Next “upgrades the model systematically” along four aspects: attention, residual, embedding and optimization. Its attention stack keeps Gated DeltaNet, a form of linear attention that compresses historical information, and pairs it with Qwen Sparse Attention (QSA), a new design that uses a lightweight compressed indexer to pick the relevant context and substantially cuts attention costs on long sequences. The model card adds that QSA selects micro-blocks rather than individual tokens; its indexer is built as MQA with 4 query heads and 1 shared key head and works within a budget of “512 blocks or 2048 tokens”.
The layout is Qwen3-Next’s: 48 layers in twelve repeats of three Gated DeltaNet layers and one attention layer, each followed by MoE. QSA takes the attention slot that Gated Attention held in Qwen3-Next, so Gated DeltaNet keeps three of every four layers.
Gated Residual widens the data pathways between layers and strengthens cross-layer information flow and training stability. N-gram Embedding adds capacity with minimal extra computation and accounts for the 51B parameters on top of the main model, while the Muon optimizer makes large-scale training more efficient.
Qwen3-Next introduced the Gated DeltaNet and Gated Attention pairing, and that design went on to run through the Qwen3.5, Qwen3.6, Qwen3.7 and Qwen3.8 series. Qwen3.8-Flash “plays the same role” for Qwen4, Alibaba says.
Up next
A harness pushes Claude Opus 5 past 30% on ARC-AGI-3
30% alone, 99.95% wrapped in an agent harness built with Strands: that’s Claude Opus 5 on ARC-AGI-3, per the author’s write-up, though NVIDIA reports 100% for the same model inside its own AVO harness.
Related stories
Comments
No comments yet. Be the first.
Join the conversation
Sign in to leave a comment. It appears after moderation.

