Microsoft AI's MAI-Base-2 trained on 16K GPUs with Muon

Microsoft AI's optimizer step for its newest base model takes 165 ms, or 2.1% of an average training iteration. Microsoft AI's own estimate of the theoretical floor is 55 ms, and the company lists exactly what accounts for the difference. Training write-ups rarely include that kind of detail. This one comes with a large claim.

In its Microsoft AI post on frontier pre-training, the company says MAI-Base-2 outperforms every open-weight model it tested on held-out evaluations. It also says the model is 4–12x more pre-training FLOPs-efficient than DeepSeek-V4-Pro-Base, which it calls the strongest open base model, and that it spent the savings on training a larger model. According to Testing Catalog's Threads post, MAI-Base-2 pre-training is complete and the MAI-Base-3 training cycle is already being prepared.

At a glance

• Microsoft AI says MAI-Base-2 is 4–12x more pre-training FLOPs-efficient than DeepSeek-V4-Pro-Base and beats every open-weight model it tested on held-out evaluations.
• For this model, Microsoft AI replaced AdamW with the Muon optimizer. After tuning, Muon's step took 165 ms (2.1% of an average iteration), against an estimated theoretical peak of 55 ms on 16K GPUs.
• A multi-round scheduler for mixture-of-experts layers keeps memory use fixed no matter how unevenly tokens are spread across experts.

Timeline

• March 19, 2024: according to Wikipedia, Microsoft AI is founded, with Mustafa Suleyman as CEO and Karén Simonyan as Chief Scientist.
• August 28, 2025: Wikipedia records the announcement of MAI-Voice-1 and MAI-1-preview, the first in-house models. Before that, Microsoft's products had usually used OpenAI's models.
• November 6, 2025: per Wikipedia, MAI announces a superintelligence division and team led by Suleyman.
• March 2026: Wikipedia says a separate Copilot division is created under Jacob Andreou, and Suleyman stays on to oversee MAI Superintelligence.
• April 2, 2026: MAI releases MAI-Transcribe-1, MAI-Voice-1 and MAI-Image-2. Tech Insider called this the clearest break yet from Microsoft's $13 billion OpenAI partnership.
• October 1, 2026: we reported that MAI-Transcribe-2-Streaming, MAI-Voice-2.1 and MAI-Voice-2.1-Flash were coming to Microsoft's MAI Playground and APIs. No prices or dates were given.
• October 9, 2026: Microsoft-Decision-1 launched. It is a post-trained Qwen3.5-9B, and Microsoft plans to rebase it on MAI and OpenAI models.
• October 12, 2026: Microsoft said Maia 200 serves Microsoft Copilot in production and that Maia 300 reaches production in 2027.
• October 15, 2026: Microsoft AI publishes its pre-training write-up for MAI-Base-2.

Up to now, almost every step on this list was a voice, transcription, image or scoring model, or a chip. MAI-Base-2 is a base language model, and the post covers how it was trained in as much detail as what it scores. Figure 1 in the post compares it with other models that have at least one trillion total parameters across a range of pre-training evaluations, where lower is better.

Muon replaced AdamW and costs 2.1% of each iteration

Microsoft AI says Muon gave more stable training dynamics and large efficiency gains over AdamW as it scaled up to MAI-Base-2. The difficulty is in how Muon works. It orthogonalizes each weight matrix's momentum using Newton–Schulz, an iterative algorithm built from several matrix multiplications, and that algorithm needs the whole matrix. ZeRO, the memory-saving scheme most large runs use, splits momentum across the data-parallel GPUs, so no single GPU holds a complete matrix.

Microsoft AI's fix, which it describes as following prior work, is to reshard during the optimizer step. Each GPU updates its local FP32 momentum shard and casts it to BF16. An all-to-all, batched over parameters with the same shape, then gives each GPU ownership of some complete matrices. Each GPU runs a symmetry-aware Newton–Schulz kernel on those matrices locally. A second all-to-all sends the updates back to their original shards, where they are cast back to FP32 and applied. Think of a stack of jigsaw puzzles split among many tables: each table briefly takes a few complete puzzles, solves them, and returns the pieces.

For the 16K-GPU setup, the team prefetches multiple batches so communication overlaps computation. It also caches resharding metadata, which the post says is especially useful in sparse MoE models with thousands of identically shaped parameters. Some weights, such as the MoE router, are replicated rather than sharded. For these, the first exchange is skipped and an all-gather runs after Newton–Schulz. The result is the 165 ms figure. Microsoft AI attributes the gap to its 55 ms communication-bound estimate to stragglers, kernel dispatch overheads, load imbalance, and memory copies and allocations during resharding.

The idea itself is not new. The Moonshot AI paper "Muon is Scalable for LLM Training" on arXiv (Feb 2025) reported roughly 2× computational efficiency over AdamW under compute-optimal training. It trained Moonlight, a 3B/16B-parameter MoE, on 5.7T tokens. NVIDIA's documentation says Muon is applied to linear layers while Adam handles embeddings and layer norms.

Microsoft AI pairs Muon with activation gradient clipping. Standard clipping scales down a whole batch's gradient when its global norm spikes, even though the outliers usually come from a small part of the data. Clipping activation gradients at selected points in the network targets those outliers more narrowly. According to the post, the two together keep MAI-Base-2's gradient norm in a much tighter range than MAI-Base-1's (Figure 7).

Expert groups keep memory fixed when tokens pile up

In a mixture-of-experts model, a router sends each token to a few experts, and some experts inevitably receive more tokens than others. Microsoft AI groups the experts on each expert-parallel rank and runs dispatch → compute → collect rounds with a fixed token capacity per expert. Overflow tokens go into extra rounds, and an overloaded expert can borrow unused capacity from others in its group. The backward pass recomputes per group on the same schedule, so memory stays static however skewed the routing gets. Raising capacity costs memory, and lowering it costs extra rounds. In practice, capacity is set so that steady state needs a single round.

All rounds are pipelined, so only the first dispatch and the last collect are left uncovered by expert compute. The forward pass fills that gap with weight quantization and shared-expert compute. In the backward pass, communication is fully overlapped with recompute and weight-gradient work. Four further changes reduce per-expert overhead: grouped GEMMs and NVLink transfers, GPU-initiated all-to-all using device-resident metadata, symmetric memory for direct NVLink token exchange, and one batched load-balancing reduction across all layers. Microsoft AI says these let it train sparser models without significantly degrading MFU (model FLOPs utilization).

What we don't know

The material we have does not give MAI-Base-2's parameter count, its token budget, or whether its weights or an API will be released. The held-out evaluations behind the efficiency claim are Microsoft AI's own, and the material does not explain what determines where a comparison falls within the 4–12x range. The engineering section is detailed enough to check against public work like Moonshot's. The headline comparison with DeepSeek-V4-Pro-Base is not, and we would treat it as Microsoft AI's claim until someone outside can run the same evaluations.

The next step in this thread is MAI-Base-3, whose training cycle Testing Catalog reports is already being prepared. No date has been announced. The earlier Decision-1 plan to rebase on MAI models is the other thing to watch, since it would show whether a model like MAI-Base-2 ends up under Microsoft's own products.