Maia 300 promises Microsoft 40% more tokens per megawatt

The number that tells you most about Microsoft's Maia 300 is a memory figure, not a compute figure: eight HBM4 stacks deliver 18.0 TB/s of peak bandwidth from a 950-watt chip. According to Microsoft's announcement, Maia 300 enters production deployments in 2027, when the Maia family scales to multi-gigawatt deployments and becomes the primary inference system for Microsoft products and services.

At a glance

• Maia 200, the current generation, already serves Microsoft Copilot in production and supports the MAI team, and GitHub Copilot and Microsoft Foundry are due to follow by the end of the year.
• Microsoft expects Maia 300 to deliver 40% more tokens per megawatt than leading GPU-based systems, and up to 7x Maia 200's rack-scale throughput in a full 96-accelerator configuration.
• Microsoft does not name the GPU systems behind either efficiency comparison, and the Maia 300 figures are projections for hardware that only reaches production in 2027.

If you have not followed the program: Microsoft unveiled the first Maia accelerator alongside the Arm-based Cobalt CPU at Ignite in November 2023. According to Microsoft, it called the chips the "last puzzle piece" for end-to-end infrastructure. Maia 100 was built on TSMC N5 with 64 GB of HBM2E at 1.8 TB/s. Maia 200, announced in January 2026, moved to TSMC 3nm, with 216 GB of HBM3e at 7 TB/s and a 750W TDP.

Maia 200 already serves Copilot and is deploying on three continents

Maia 200 is serving Microsoft production workloads, Microsoft Copilot among them, and supports the MAI team as it develops next-generation models. Across those workloads it runs OpenAI models, Microsoft's own MAI models and leading open-source model families. Microsoft says it delivers up to 1.7x more tokens per watt than leading GPU infrastructure for built-in AI capabilities in Teams, Copilot and Cowork.

The chip is deploying across the United States, Europe and Australia. According to Inside Deep Tech, as of September 2026 it was already in Microsoft's US Central fleet near Des Moines, with US West 3 near Phoenix next.

Microsoft has lined up more work for the two generations together: multi-step agentic work for GitHub's coding experiences, cloud-scale model serving for Microsoft Foundry, inference and data generation for the MAI team, and latency-sensitive interactions for Dragon Copilot, which is meant to reduce clinicians' administrative burden.

A Maia 300 rack puts 96 accelerators inside 167 kilowatts

Maia 300 is the third generation of the program, and Microsoft calls its design inference-first: built for long-context, memory-intensive and decode-heavy workloads such as long-context reasoning, latency-sensitive conversation and sustained multi-step agentic tasks. Each chip carries eight HBM4 stacks with 18.0 TB/s of peak bandwidth within a 950-watt TDP.

The headline numbers come from the rack. Ninety-six accelerators share a 167-kilowatt power envelope, 27.6 TB of HBM capacity and 1.7 PB/s of aggregate HBM bandwidth across one high-speed scale-up domain. In that full 96-accelerator configuration, Microsoft says, a rack delivers up to 7x the rack-scale token throughput of Maia 200.

The silicon is split into chiplets, with separate compute, memory-controller and I/O dies, and the compute chiplets are made on TSMC's N2P process node. For comparison, Maia 100 was a single reticle-size die of about 820 mm² on N5, provisioned at 500W with a design TDP of up to 700W.

Maia runs only inside Azure, separate from the rented MI300X and H100 VMs

Maia is Azure-exclusive silicon, co-optimized with Microsoft's racks, liquid cooling, Ethernet fabrics and software. It is not the same product as the ND MI300X or ND H100 GPU virtual machines Azure rents out. According to Microsoft, when Maia was first announced, Scott Guthrie said the goal of in-house chips was to "optimize and integrate every layer of the infrastructure stack to maximize performance, diversify our supply chain and give customers infrastructure choice."

According to Inside Deep Tech, software for Maia centers on PyTorch, Triton and Maia-specific APIs, but CUDA still decides the choice for many teams. Maia wins for Azure first-party inference and loses when a team needs rentable multi-cloud GPU portability. Back in 2023, SemiAnalysis wrote that Microsoft trailed Google and Amazon in deploying custom silicon in its datacenters.

Maia 300 uses on-die SRAM and MXFP4 to move less data

Microsoft's premise is that in large inference systems, moving data efficiently, rather than raw compute, is increasingly the bottleneck. Maia 300 pairs a software-defined dataflow architecture with AI-assisted kernel optimization to keep data local and keep compute, memory and networking busy. Specialized kernels use on-die SRAM as a software-managed scratchpad, which reduces pressure on HBM.

Picture a cook who sets out everything for the next few steps on the counter instead of walking to the pantry each time. The knife work is no faster, but much less time goes to fetching. Low-precision formats, including an enhanced MXFP4, shrink both the computation and the KV-cache, the attention state a model stores for every token of context, so larger context windows fit in the same power envelope.

Networking is treated as part of the same dataflow. Interconnects between accelerators are tuned for tensor parallelism, and a custom AI transport over Ethernet-based infrastructure gives one topology that scales from a single rack to a whole datacenter. According to Inside Deep Tech, Maia 200 also uses Ethernet rather than an NVLink-style proprietary fabric.

Every efficiency figure here comes from Microsoft itself and is measured against GPU systems it does not name, and Maia 200's 1.7x covers only built-in AI features in Teams, Copilot and Cowork. Oddly, Maia 300's 40% more tokens per megawatt looks smaller than Maia 200's up to 1.7x more tokens per watt, which likely reflects a different baseline that Microsoft does not specify.

Dates on the Maia calendar

Microsoft Ignite 2026 takes place next month in San Francisco and online, and Microsoft has promised more on its strategy there. The next checkpoint is the end of the year, when GitHub Copilot and Microsoft Foundry are due to run on Maia 200. Maia 300 production deployments follow in 2027. Microsoft has not named the GPU baselines, and it has not said how many chips "multi-gigawatt" means.