Skip to content

Open models

Kimi K3.1 Pro weights land on HuggingFace at 2.8T params

5 min read

Our AI analyst checked this story itself: it ran, measured and compared against the originals. What we checked

Written by an AI model from the linked sources and checked against them by a second model. How we work

Image: aicoder.com

Moonshot AI put the weights of Kimi K3.1 Pro, its 2.8T-parameter Mixture-of-Experts model, on Hugging Face on November 4 under a custom “Kimi K3.1 Pro” License. The checkpoint is 1,454 GiB, small enough for the weights alone to fit on one 8×B200 node. As of November 5, though, no released version of vLLM or SGLang runs its architecture, so self-hosting currently depends on day-0 Docker images and development branches.

The model was post-trained from Kimi K3. It activates 104B parameters per token, takes text and images, and has a 1,048,576-token context window. In Moonshot’s benchmark table it scores 65.7 on Terminal-Bench 4.0, against 66.4 for Claude Opus 5.5. That table is Moonshot’s own work, though. Several of its benchmarks are internal, and Kimi was usually run in the Claude Code harness while many rival scores were taken from leaderboards and other labs’ reports.

Getting it to run

The model card recommends vLLM, SGLang and TokenSpeed. The vLLM recipe requires vLLM 0.34.0 or later, but the newest release on PyPI is 0.33.0, from November 2. Until a new release ships, the route is the day-0 images vllm/vllm-openai:kimi-k31-cu134 (CUDA 13.4) and vllm/vllm-openai-rocm:kimi-k31, both pushed on November 4. The same day, SGLang’s main branch added “basic Kimi-K3.1 support”. Its latest release, 0.5.23 from October 30, doesn’t have it, and the cookbook builds from a kimi-k3.1-day-0 branch. The card’s link to a K3.1 Pro section on TokenSpeed’s recipes page goes nowhere: when we fetched the page, it covered Kimi K3 only.

The vLLM recipe’s baseline hardware is one B300 node, one MI355X node or two GB300 trays, with tensor parallelism across eight GPUs. The long context costs less memory than its size suggests. Most of the 93 layers (69 of them) use Kimi Delta Attention, a linear-attention design whose per-sequence state stays the same size however long the input gets. By our count from the config, those layers hold a fixed 0.44 GiB per sequence. The other 24 layers keep a cache that grows with context. At the full million tokens it comes to about 33.0 GiB per sequence in BF16, or 16.5 GiB in FP8. Engines allocate differently: vLLM documents storing the latent in NVFP4 at 352 bytes per token per layer, down from 576.

None of this applies to the hosted route, where kimi-k3.1-pro is served on platform.kimi.ai through OpenAI- and Anthropic-compatible APIs. Two request details apply whichever way you run it. The top-level reasoning_effort field takes “low”, “high” or “max” and defaults to “max”. And in multi-turn conversations and tool calls, the model needs the complete assistant message the API returned to be passed back.

What the license asks

The “Kimi K3.1 Pro License” is the Kimi K3 License with two lines changed: the title and the model name that has to appear on screen. It is MIT-style with two conditions. A licensee that runs a Model-as-a-Service business and whose revenue, counting affiliates, exceeds $20 million over any 12 consecutive months must sign a separate agreement with Moonshot before any commercial use. Products with more than 100 million monthly active users or more than $20 million in monthly revenue must prominently display “Kimi K3.1-Pro” in their interface. Internal use and Moonshot’s certified inference partners are exempt. The serving clause first appeared with K3, whose weights came out on July 27. K2.5, K2.6 and K2.7-Code used a Modified MIT License whose only condition was the display rule.

Reading the benchmark table

Kimi’s scores use max effort, temperature 1.0 and top-p 0.95 unless a footnote says otherwise, and BrowseComp, for one, ran at top-p 1.0. On Terminal-Bench-Science, Moonshot’s own run of DeepSeek-V4.1-Flash-1017 scored 9.3, but the table instead lists 15.7, a leaderboard score for the older DeepSeek-V4.1-Flash. K3.1 Pro falls well behind on HLE-Diamond, at 39.4 against 60.9 for Opus 5.5. On ExploitBench (pass@5) it scores 74.7, against 91.0 for Opus 5.5 and 100 for GPT-6 Astra. In October, Kimi said it had removed cyber-related data from mid-training and reinforcement learning ahead of the open release.

Our analysis

What we checked

On November 5 we pulled the repo’s file sizes, config, license and the headers of all 96 safetensors shards, compared them with the Kimi K3 repo, and checked the recommended engines’ releases and recipes.

How we check stories

Moonshot’s architecture numbers hold up: by our count the model has 2.78T parameters, and 104.6B are active per token. The card says K3.1 Pro replaces K3’s full-attention layers with sparse ones. Judging by the tensor shapes, K3’s attention weights are unchanged and 24 small indexers have been added on top of them. For anyone already serving K3, that adds only about 357 MB of weights. The repo went public on November 4, but its only commit is dated November 2.

GPUGPUs needed for weights8-GPU nodes
B200 (192 GB)8, with 81.9 GiB spare1
H200 (141 GB)112
H100 (80 GB)193
  • These counts cover weights only. KV cache and activations come on top, and multiple nodes are required only on Hopper GPUs.
  • The routed experts account for 2.72T parameters (97.9%) and are the only part stored in MXFP4. The remaining 57.4B parameters are in BF16, which averages out to 4.5 bits per parameter.
  • The sparse-attention layers sit at every fourth layer, plus the final one. The vision tower has 401.2M parameters, and its projector adds 46.1M.

No comments yet. Be the first.

Join the conversation

Sign in to leave a comment. It appears after moderation.