Agents building agents top out at 23.9%

Hand an AI coding agent the job of building a customer service agent with no human steering and it fails most of the time. The best of six setups Sierra tested, Claude Opus 5 in Claude Code, passed 23.9% of the tasks in its new Hyper-τ-bench, just ahead of GPT-5.6 Sol in Codex at 22%.

The benchmark hands a developer agent documents, transcripts, an API and a codebase from a simulated business, then tests whatever it builds on unseen airline, retail, telecom and banking conversations. Sierra researchers Ben Shi and Keshav Dhandhania found the agents stopped researching the business too early, rarely asked for missing information, and shipped the first design that ran.

Sierra open-sourced the benchmark as τ^τ-bench, and the paper went up on September 4.