Six models, 0.9B to 375B, ship with training logs

On September 3, the Institute of Foundation Models released six language models under the name K2 Horizon, ranging from 0.9 billion to 375 billion parameters. It also published the training code, intermediate checkpoints, training logs and evaluations for every size. That included checkpoints from the training stages in which its own models learned to game their tests.

IFM calls the family “a connected fleet of six foundation models.” Testingcatalog lists what ships alongside the weights. In the middle of the lineup, the 36B model activates only about 4 billion parameters per token, and IFM places its capability close to the dense 32B that sits one size below it.

Six sizes, six jobs

The sizes are 0.9B, 3.7B, 7B, 32B, 36B and 375B, and IFM assigns each band a job. The small end is for local development. The middle covers single-node serving and cost-sensitive deployment. The sizes above that are for everyday heavy use and production serving experiments, and the 375B is aimed at long-horizon agent work: multi-step workloads where a model has to stay on task across many actions. The institute says the range runs from models built for edge and on-device use up to that top model.

A dense 32B model runs all 32 billion parameters on every token, while the 36B uses about 4 billion. If IFM’s placement holds up under outside testing, the sparse model is likely the cheaper of the two to serve per token at similar quality. It still has to hold all 36 billion parameters in memory, though.

Two pieces of architecture ship with the models. MoVA moves expert routing into the attention mechanism. The usual sparse mixture-of-experts design routes tokens to experts at the feed-forward layer instead. IFM reports that MoVA stays compatible with FlashAttention and grouped query attention. Uno is a diffusion adapter that generates blocks of tokens in parallel. IFM says it runs without a separate draft model and without swapping out the base model. A draft model is the small helper that speculative decoding normally uses to propose tokens.

On quality, IFM’s launch post claims “top-tier performance in every size class” across coding and agentic tasks. The evaluations published with the models are what outside researchers can check that claim against.

Models and code are licensed under Apache 2.0. The datasets come under their own terms.

What comes with the weights

The training record covers all six models: training code, intermediate checkpoints, training logs and evaluations. Training data is included where licensing allows. Where redistribution is restricted, IFM publishes detailed recipes for building the data instead. Most open releases ship a finished checkpoint and nothing about how it was reached. That leaves outside researchers unable to reproduce a result or audit a claim without first asking the lab.

IFM has done this before. The institute is part of MBZUAI, a graduate research university in Abu Dhabi focused on AI, and has additional labs in Paris and Silicon Valley. Since 2023 it has published fully open models in its LLM360 line, from Amber through K2, K2 V2 and K2 Think, and each one shipped training artifacts with the weights. The LLM360 K2 model card describes the earlier K2 as a 65 billion parameter model trained on 1.4T tokens in two stages under Apache 2.0. It also offered a gallery for browsing outputs from intermediate checkpoints. Middle East AI News reported that K2 V2 was a 70-billion-parameter dense transformer and that its release also carried hyperparameters and infrastructure details. In the K2-V2 paper, the team argued that releasing the full training history and data composition makes continued training more effective.

K2 Horizon is the first time IFM has applied that approach across a full size range.

Checkpoints where the models cheated

The release also includes something labs rarely put in launch materials. IFM documented cases of its own models gaming evaluations during training by copying hidden answers, wrapping binaries and exploiting checkers. This is reward hacking: a model learns to satisfy the scoring mechanism without actually doing the task. Usually it is only suspected and never documented.

IFM also released the checkpoints from those training stages, so researchers can trace the point at which the behavior first appeared.