Aleph Alpha puts a 1M-context model on your own servers

Kolibri, the model Aleph Alpha released on October 3, uses 3.46 billion of its 78.1 billion parameters on each token. The German lab says that is enough to match models with up to four times as many active parameters on math, code, grounding, agentic and long-context tasks. The full weights are on Hugging Face under Apache 2.0.
The model handles English and German and is aimed at government and regulated industries that would rather run it on their own hardware than send internal data to an outside inference service. A post on Threads announcing the release quoted the model’s description: “The model supports an explicit reasoning mode and tool calling. It is optimized for long-context and inference efficiency.” The model card lists it as Kolibri-1.
How the sparsity works
Kolibri is a Mixture-of-Experts transformer: each layer holds many small feed-forward blocks, the experts, and a router picks a few of them for every token. Kolibri has 384 experts and sends each token through six. So the memory bill covers all 78.1 billion parameters, while the compute per token covers only the active 3.46 billion.
Attention is split the same way. Only 10 of the 50 layers use full attention over the whole context; the other 40 use a 512-token sliding window, so in those layers each token looks back at just the previous 512. Aleph Alpha says the split is there to contain inference costs, since the window layers do not grow with the prompt.
Long-context adaptation reached 256,000 tokens. Aleph Alpha still ships settings to serve the model at 1,048,576 tokens, beyond the length it was adapted on.
German makes up 21.3% of the pre-training tokens. The vocabulary has 128,000 entries covering both languages, and the tokenizer is built to keep German compound words intact. The model offers four reasoning settings: none, low, medium and high.
The training run
Testing Catalog reported that Kolibri was trained on 768 Nvidia B200 GPUs. Pre-training covered 20 trillion tokens over 21 days, followed by mid-training and long-context adaptation, bringing the total to nearly 24 trillion. The model builds on the earlier Kolibri Origin and on Aleph Alpha’s automated Model Factory pipeline, and was built in Germany and trained in Germany and Finland.
Aleph Alpha says it specialized the model for German, math, coding, long-context work and agentic tasks. It also built evaluation suites for public administration, automotive, semiconductors, industrial technology and aerospace, and says none of them use customer data.
The vendor’s scorecard
All the published scores come from Aleph Alpha, run on its own harnesses and at the highest available reasoning setting where one applies:
- English overall: 75.5
- German overall: 70.8
- AIME 2025: 96.9
- LiveCodeBench v6: 85.9
- BFCL v4 overall (function calling): 61.4
The company places Kolibri on the quality-versus-serving-cost Pareto frontier in both languages, meaning no model it compared delivers more quality for the same serving cost. The consultancy innFactory, summarizing the tech report, said the two overall scores put Kolibri ahead of Qwen3.5/3.6 35B-A3B, Gemma 4 26B-A4B and Nemotron 3 Nano in the same size class, but that its closed-book knowledge and multi-turn tool calling fall below those models, and that it is designed to work with retrieval. The same summary gives a knowledge cutoff of June 18, 2026, and notes the model is text-only. No independent evaluations were available at launch.
Trained to abstain
Kolibri was trained on abstention examples and with the company’s Merlin-Arthur procedure, an adversarial training setup that teaches the model to withhold an answer when the supporting evidence is absent.
On AA-Omniscience, Kolibri avoided a wrong answer on 44% of items. Kolibri Origin managed 14.8%. On the company’s own M/A grounding score, Kolibri reached 0.23.
What it takes to run
The FP8 weights come to about 78 GB, innFactory said, so two 80 GB H100 or A100 cards, a single H200 or a single B200 are enough to host the model.
The licence is a change for Aleph Alpha. When it launched its PhariaAI enterprise platform on August 26, 2024, it released two 7B Pharia-1 models with training code open only for non-commercial research and educational use. innFactory describes Pharia-1-LLM-7B as a dense model with 8,192 tokens of context under the non-commercial Open Aleph License; Kolibri’s Apache 2.0 terms allow commercial use without a separate licence agreement.
Outside hosting has already started. Konark Modi posted that his team had put Kolibri-1 up for anyone to try free for a limited period, noting that the release fell on German Unity Day.
For self-hosting, Aleph Alpha’s route goes through its own inference package and a Kolibri-specific vLLM plugin, which carries the parsers for the model’s reasoning output and tool calls.
Up next
Attackers hit a Mythos-found HFS bug within a day
Servers running Rejetto HFS in the US and Japan drew attacks from a China-hosted IP a day after a researcher published an exploit for a bug Mythos found. Of 286 Mythos-linked CVEs, it’s the second known to be exploited in the wild.
Related stories
- Recursive opens Flint, a 40B coding model with 2.5B active
- Nemotron 3.5 Super VL gives NVIDIA’s 120B agent model eyes
- Liquid AI’s 2B-active model beats Qwen on LiveClaw
- Six models, 0.9B to 375B, ship with training logs
- Kimi K3.1 Pro weights land on HuggingFace at 2.8T params
- Tencent ships Hy4 at Hy4 preview’s API price
Comments
No comments yet. Be the first.
Join the conversation
Sign in to leave a comment. It appears after moderation.

