Liquid AI’s 2B-active model beats Qwen on LiveClaw

Liquid AI’s LFM2.5-24B-A2B uses 2.45 billion of its 24 billion parameters on each token. On Liquid’s own 16-benchmark table it splits exactly even with Qwen3.6-35B-A3B: eight wins, mostly in tool use and instruction following, and eight losses, including coding and science. The open-weight reasoning model went up on Hugging Face on October 13, and Liquid AI’s model card pitches it for “fast, private agents on-device.”
It is the largest model in the LFM2.5 family, and Liquid says it is also the last. The family now runs from LFM2.5-230M up to this one. The card’s headline rounds the active count down to 2B, while its spec table gives 2.45B.
How 24B parameters run like 2B
The network has 40 layers. Thirty of them are double-gated short convolution blocks. A short convolution mixes each token only with a small window of its neighbours, and gates control how much of that signal passes through. Its cost doesn’t grow with the length of the conversation. The other 10 layers use grouped-query attention, the standard mechanism that looks back across the whole context. Only those ten keep a cache that grows with every token, which is where most of the memory goes on long agent sessions.
The second saving comes from mixture of experts. Each MoE block holds 64 experts, and a router sends every token to four of them. Most of the weights sit idle on any given step, which is how 24B in total comes down to 2.45B doing the work. Liquid has described the hybrid design this way since the first LFM2.5 release: fast inference on CPUs with a low memory profile, as the building block for on-device agents.
The cheap step matters more here because the model always reasons before it answers, so it produces many tokens the user never sees. With roughly 2B active parameters, “each reasoning token is cheap,” the card says.
What changed from LFM2-24B-A2B
The architecture is carried over from LFM2-24B-A2B. The changes are in the window, the tokenizer, the training data and the answer mode:
- Context window: 32K → 131K tokens (131,072 exactly)
- Vocabulary: 65K → 128K tokens, through an in-place tokenizer expansion aimed at better non-Latin tokenization
- Pre-training data: ~17T → ~33T tokens
- Answer mode: LFM2-24B-A2B was not a pure reasoning model → LFM2.5 always thinks, and its chat template inserts a
<think>tag at the start of every assistant turn
Between pre-training and post-training sits a mid-training stage. It used 2T tokens at 32K context, weighted toward reasoning, math, code, instruction following and multilingual text, then 300B tokens of long-context data at the full 131K. Tool calls and multi-turn trajectories appear in both phases. Post-training stacks supervised fine-tuning, teacher specialization and two on-policy distillation stages, one non-agentic and one agentic. It ends with agentic reinforcement learning inside four popular harnesses: OpenClaw, Hermes Agent, Pi and OpenCode. The goal, per Liquid, was for the model to learn “their tools, system prompts, and interaction patterns.”
The model is text-only and lists 16 languages, among them Arabic, Hindi, Thai, Vietnamese, Russian and Polish alongside the usual European and East Asian set.
Where it beats Qwen, and where it doesn’t
Every score in the table comes from Liquid’s own runs. All four models were served through vLLM, and each benchmark had its own sampling settings. The comparison set is Qwen3.6-35B-A3B, Gemma-4-26B-A4B-it and Nemotron-3.5-Lightning (30B-A3B).
The wins are clustered in agent work. On AutomationBench, LFM2.5 scores 27.69, against 8.72 for Qwen, 5.46 for Gemma and 2.61 for Nemotron, with up to 50 turns per task. On LiveClaw it scores 48.09 to Qwen’s 38.19, with Gemma at 41.25. That run used temperature 1.0, the vLLM default because the agent sets none, and capped output at 8,192 tokens. ToolSandbox goes 80.71 to 75.02. On τ³-Bench Banking the lead is 8.04 to 7.63, though every model scores in single digits there. On BFCLv4, the function-calling leaderboard, the win is 65.01 to 65.00.
Instruction following is the other strong area: 96.75 against 93.06 on IFEval, 74.69 against 64.21 on IFBench, and 84.38 against 81.53 on Multi-IF.
The losses are where the work needs knowledge or long chains of code. On SWE-Bench Verified, run in the OpenCode harness, LFM2.5 scores 51.00 against Qwen’s 66.20, and it also trails Nemotron’s 53.40. OpenCode is one of the four harnesses the model was trained in with reinforcement learning, so the 15-point gap is hard to blame on unfamiliar tooling. WebArena Verified, with up to 40 browser actions, shows 26.75 against 37.62, and Nemotron’s 28.69 is ahead there too. GPQA Diamond goes 74.42 to 83.69, SciCode 32.71 to 37.35 and PinchBench 83.18 to 89.93. HLE (7.40 to 7.69) and MATH-500 (96.04 to 96.73) are near ties that still go Qwen’s way.
AA-Omniscience, where scores go negative, puts LFM2.5 at -24.90, next to Qwen’s -24.35 and Gemma’s -59.39, while Nemotron reaches -3.58. Liquid’s own advice fits that result: it recommends against using the model alone for knowledge-heavy tasks and suggests pairing it with retrieval or web search, “which plays to its tool-calling strength.” The card for the smaller LFM2.5-8B-A1B carried a similar warning about knowledge-intensive questions without retrieval and about heavy programming.
184 tokens a second, under which conditions
The laptop numbers come from Pipette, Liquid’s in-house device benchmarking setup. They were measured with llama.cpp, Q4_K_M quantization and a 4K-token input context. Under those conditions LFM2.5-24B-A2B decodes 184 tokens/s on an Apple M5 Max and 114 tokens/s on an AMD Ryzen AI Max+ 395. Liquid says that is up to 1.7x faster than the next-best of the compared MoEs, with the lowest memory use of the group, and that it also reads prompts faster than the others.
The server number is a different kind of measurement. Over 21K output tokens per second on a single NVIDIA B300 is total throughput across many parallel requests at high concurrency, with SGLang 0.5.22 and BF16 weights. Each request had 1,024 input tokens and at most 256 output tokens, and every finished request was replaced immediately to keep the load constant. It measures how many users one GPU can serve, not how fast a single answer arrives.
A speed-up through speculative decoding is listed but not shipped. The LFM2.5-24B-A2B-DSpark draft model for SGLang is marked “coming soon.” A draft model of this kind guesses several tokens ahead for the large model to verify in one pass. For the 8B-A1B model, Liquid shipped a 328M drafter of the same type and claimed about 2.5x faster decoding with identical outputs.
Wiring it into an agent
Six checkpoints are listed, five of them available now. The base model is for fine-tuning. The post-trained model comes in native format for Transformers (version 5.0.0 or later), vLLM and SGLang. An FP8 build uses block-wise FP8 weights and dynamic FP8 activations for lower-memory GPU serving. A GGUF build targets llama.cpp and CPU inference, there is an ONNX export, and a 4-bit MLX build covers Apple Silicon. The recommended sampling is temperature 0.1, top_k 50 and repetition_penalty 1.05. Fine-tuning notebooks cover Unsloth and TRL, including LoRA variants of SFT, DPO and GRPO.
Tool calls follow a fixed format. By default the model writes Pythonic function calls, a Python list placed between <|tool_call_start|> and <|tool_call_end|> tokens. You can switch it to JSON calls with an instruction in the system prompt. Tool results go back under the tool role, and the model then writes a plain-text answer.
To run it in a harness, Liquid’s guide has you serve the model behind any OpenAI-compatible backend and point the agent at it. For Hermes, that means setting model.context_length to 131072 and agent.tool_use_enforcement to true.
Up next
OpenAI says its own models helped trace a hidden office bug
A recorder wrapped in toilet paper sat behind a TV in an OpenAI executive room, and OpenAI says its own models helped trace it to an engineer, whom the FBI then arrested. Does anyone at your company actually sweep meeting rooms for devices?
Related stories
- Recursive opens Flint, a 40B coding model with 2.5B active
- Nemotron 3.5 Super VL gives NVIDIA’s 120B agent model eyes
- Aleph Alpha puts a 1M-context model on your own servers
- Mistral Large 4 weights are out, API now from $0.70 per 1M
- Kimi K3.1 Pro lands just behind Opus 5.5 on Terminal-Bench
- DeepSeek’s minor Flash update beats Opus 5 on DeepSWE
Comments
No comments yet. Be the first.
Join the conversation
Sign in to leave a comment. It appears after moderation.

