Sai claims a top OSWorld 2.0 score at lower cost

Simular's computer agent Sai reached a 73% success rate on OSWorld 2.0, the 108-task benchmark of long desktop work, at roughly two-thirds the cost of the frontier models it outscored. The company published the figures in a benchmark update on Thursday, reported by The New Stack.

At a glance

• The benchmark's 108 tasks are measured in hours rather than minutes, covering work skilled humans typically need more than an hour to finish, such as reconciling receipts scattered across email and expense reports.
• Simular puts Sai ahead of GPT-5.6 Sol at 62.57%, as reported by OpenAI, and of Opus 5 at 70.57%, as reported by Anthropic, on the same 108-task benchmark.
• The scores are not yet on the OSWorld 2.0 leaderboard: Simular said it is submitting them and uploading its trajectories to Hugging Face, so the result currently rests on the company's own reporting.

The number that matters in this claim is not the score but the price beside it. Cost per completed task, rather than peak capability, appears to be the axis on which computer-use agents get judged once they move from demos into routine back-office work. A self-published result at two-thirds the cost of frontier models reads as an argument about that market rather than about the leaderboard.

Sai makes about 1.5x fewer model calls than pure models

Sai runs on full desktop applications and webpages, calls APIs and writes code, combining frontier and specialist models. Simular attributes the cost gap to neurosymbolic planning, which pairs neural exploration with symbolic code so a solved task is encoded as a script that replays identically.

The agent takes more actions per turn and uses roughly 1.5x fewer model calls on average than pure models, writing longer subtasks in Simulang, a Claude Code skill for desktop automation on macOS. Adaptive summarization keeps the input bounded, and the prompt prefix stays constant as long as possible before summarization.

Sai also calls specialized models to localize interface elements, to reason and to verify, avoiding expensive models where full task context is unnecessary. In a Chrome Dino test run against Sol and Opus, Sai measured the ground line, obstacle speed and its own input latency from raw pixels, then ran a screen-capture control loop on the virtual machine in a single execute call.

OSWorld 2.0 measures its tasks in hours rather than minutes

OSWorld 2.0 was launched on June 26 by the Executable Language Grounding Lab, part of the HKU NLP Group at the University of Hong Kong, and updated on August 8. Its tasks are measured in hours rather than the minutes of the first version.

The 108 tasks require reasoning across scattered data sources such as receipts spread between email and expense reports, reacting to changes such as a message arriving mid-task, following tutorials such as reimbursement guidelines, and resolving contradictory data. Task 28, vaccine booking, supplies an email about required immunizations, a scanned vaccination record on the desktop and a booking site with price, distance and date constraints.

Simular calls it the most robust open benchmark currently available and the closest to real work. The company's open-source Agent S was the first system to pass the human baseline on OSWorld 1.0 last December, before the second version raised task length past an hour.

Simular says its work spans the entire stack of a computer agent

Simular, based in Palo Alto and founded by former DeepMind scientists Ang Li and Jiachen Yang, says its work spans the full stack of a computer agent: the models, planning and grounding, the scripting layer, the virtual machines and the user interface.

Yang told The New Stack that agents doing recruitment outreach, invoice validation or news research should not cost a fortune because the underlying model was trained to solve open mathematical conjectures. "Posterity will find it ludicrous that people are still building models that way right now," he said.

Santiago Valdarrama wrote on LinkedIn that Sai's operation on a full desktop, rather than only a browser or an API, lets it handle applications that block standard automation. Baljinder Lally, who works on AI revenue systems, replied that reliability comes from guardrails, visibility and human-in-the-loop checkpoints, and AgenticMode AI founder Jahanzaib A. said voice agents that nail demos can fail silently in production.

The leaderboard entry is still in progress

Simular said it is in the process of submitting to the OSWorld 2.0 leaderboard and uploading its trajectories to Hugging Face, and it noted that companies including Anthropic and OpenAI publish benchmark results on their own sites first. No timeline was given for either step, and no per-task pricing was published beyond the two-thirds cost comparison. Sai was first released in March.