Skip to content

OpenAI

OpenAI trains GPT-6 Astra on real Ironclad contract work

4 min read

Written by an AI model from the linked sources and checked against them by a second model. How we work

All about: OpenAI models and products →
Image: openai.com

OpenAI trained GPT-6 Astra on contracting workflows built with Ironclad. On 11 research tasks the model scored 55.0% on average, against 41.6% for GPT-5.6 Sol, and it took about half the estimated time per attempt, the company said in an October 6 post on the OpenAI site. Ironclad sells AI contracting software. It is the first of what OpenAI describes as a small number of software companies it will partner with directly.

OpenAI headlines a score “32% higher,” which is a relative figure. In absolute terms Astra is 13.4 points ahead. The two models also ran at different settings: Max reasoning for Astra and High for Sol, which OpenAI says are the settings where each scored best. Estimated time per attempt dropped from 37.0 minutes for Sol to 19.2 minutes for Astra, 48% lower. These are estimates, which the post also calls simulated time, not wall-clock measurements.

OpenAI also gives a human baseline. It estimates that an experienced Ironclad user needs about 30 to 40 minutes per task on average. Sol’s 37.0 minutes sits inside that range. Astra’s 19.2 minutes is well under the low end of it.

The post walks through a typical task. A legal operations team wants a process for buying software. Finance approves purchases above a certain amount, Security reviews some requests, and Legal reviews nonstandard terms. Whoever sets this up has to turn that short list into an intake form, document templates, approval rules and a record of the final agreement. An agent doing the same job has to configure the Finance approval at the spending threshold and then check that requests on both sides of it take the right path. “Getting individual steps right is not enough: the finished process must work across the situations it was designed to handle,” OpenAI writes.

Ironclad employees, along with people at OpenAI who use Ironclad, helped pick the 11 tasks across legal, commercial and procurement work. They include setting up nondisclosure agreements, building procurement approval processes, and updating a reusable legal clause so that it reflects the jurisdiction a requester selects. Each task is scored against 8 to 50 criteria, depending on how complex it is. That lets researchers see which parts of a run a model got right and where it fell short.

The training itself took place elsewhere. Ironclad provided hosted copies of its product where the models could practice. OpenAI researchers wrote synthetic training tasks around representative workflows, generated for the purpose rather than drawn from the 11 evaluation tasks, and used reinforcement learning so the models improved through practice and feedback. OpenAI calls Astra its first frontier model trained on Ironclad tasks.

On one task that OpenAI shows in side-by-side clips, Astra met about 94% of the criteria in an estimated 20 minutes. Sol met about 85% in an estimated 32 minutes. Across all 11 tasks, though, Astra’s average still met only a little over half the requirements. And it isn’t the strongest model on this test: an internal model used during Astra’s development reached 63.7%, and OpenAI says it aims to bring those gains to future models. When Astra launched, OpenAI’s claims of state-of-the-art computer-use results went out without numbers. This evaluation comes with figures and settings, but it is still OpenAI’s own.

The section of the post on what the partnership means for Ironclad is frank about the risk. A contracting process has to handle exceptions without breaking the rules a business depends on. “If an agent loses track of one of those rules halfway through a task, that limits what a software company can confidently ask it to do,” the post says. It adds that this is why human oversight still matters, and why a full contracting platform remains essential, as agents get better at this work. In return, Ironclad gets a way to put its customers’ problems into the development of the underlying model. As models improve, it may also use them in more of its own products.

OpenAI is now asking other software companies for professional tasks that today’s agents still can’t complete reliably. The requirements are specific. A partner has to bring a concrete example of the task, evidence of where the agent fails and a way to judge a successful result. It also needs people who know the work deeply, a secure environment for testing, and data that can safely be used for research.

No comments yet. Be the first.

Join the conversation

Sign in to leave a comment. It appears after moderation.