GPT-6 Astra hits 66.7% on wet-lab tasks, GPT-5.6 Sol 43.8%

In one cloning task, GPT-6 Astra noticed that the negative-control plate had “LB Kan50” written on its rim, which means kanamycin in an experiment run on ampicillin, and rejected it as a control. GPT-5.6 Sol saw no colonies on the same plate and built its conclusions on that. The task is one of 100 in BioBasicsBench, a wet-lab analysis benchmark OpenAI published on November 6 with Ligo, an enzyme-design company. On it, Astra earned 66.7% of the available credit at the xhigh reasoning setting, against 43.8% for GPT-5.6 Sol. Those percentages are average rubric credit, partial answers included, from a benchmark OpenAI ran only on seven of its own models. They rank those models on routine lab analysis. They are not a rate of correct lab decisions.

What the agent gets

Scientists wrote each task from data out of their own experiments and paired it with an expert-written grading rubric. Every task then went through three reviews: one by a Ligo scientist who had not written it, one by an outside PhD-level scientist working in a wet lab, and a final internal pass. Reviewers checked the prompt, the evidence, the rubric, the author’s notes and a graded model attempt. The tasks cover five areas: protein methods, imaging and cytometry, quantification and assays, nucleic acid methods, and molecular cloning. They run from reading the molecular weight of a DNA sample off a gel to detailed assay analysis.

The evidence comes in the raw form a lab produces. Gel and blot images appear in 31 tasks. Protocols, lab notebooks and photographed handwritten notes appear in 27, and CSV and Excel exports and raw flow cytometry files in 20. Microscopy shows up in 14, and seven tasks include videos of bench work such as streaking plates or weighing on a balance. A single task can mix several kinds of data, and reference sheets such as ladder charts and plasmid maps are included as supporting material.

The agent returns a structured answer: the conclusion, measurement or recommendation the task asks for, plus reasoning where required. Each rubric criterion carries its own credit, so a correct conclusion scores differently from a partly correct analysis. Depending on the task, the criteria cover the conclusion, the numerical results, the reasoning, or recognizing that the evidence is insufficient. Most tasks carry several overlapping skill tags: 88 are tagged for tacit biological knowledge, 88 for reading images and data, 68 for error detection, 50 for protocol understanding and only 19 for spotting insufficient data, even though OpenAI features that last skill in its own description of the benchmark. OpenAI says BioBasicsBench shows where models need to improve and helps it track what those gains mean for scientific progress, biosafety and biosecurity.

Reading the scoreboard

OpenAI tested seven models across 37 combinations of model and reasoning effort, running each task five times per combination, for 18,500 results in total. The 95% confidence intervals come from resampling whole tasks, which keeps each task’s five attempts together. Xhigh is the highest effort level reported for all seven models. Max-effort runs, for the models that have them, appear only in OpenAI’s chart.

At xhigh, Astra’s 66.7% has a 95% interval of 60.3–73.1%, and GPT-5.6 Sol’s 43.8% has one of 37.6–50.0%. Those two ranges don’t overlap. GPT-6.1 Sol is a different story: its 62.0% (55.6–68.4%) sits largely inside Astra’s range. And not every GPT-6 model beats the older line. GPT-6 Luna scored 38.3% at xhigh, below GPT-5.6 Sol, while generating 13,120 output tokens per task. GPT-5.2 came last with 31.2%.

Astra also produced less output to get its score: 4,704 output tokens per task at xhigh, roughly half GPT-5.6 Sol’s 9,099. That number counts generated output only. It says nothing about total compute or what a run costs. OpenAI says Astra beats GPT-5.6 Sol by a wide margin even at low reasoning effort, and that these gains suggest stronger native multimodal capabilities may be key to everyday wet-lab analysis.

Two of OpenAI’s examples show what that improvement looks like. In one, GPT-5.6 Sol classified cells as dead from the signal of a dye that marks dead cells. But that detector’s window (470–540 nm) overlaps with a second dye’s window at 525–540 nm, so brightness alone couldn’t settle it. Astra flagged the ambiguity and asked for better controls. In the other, a microscope video showed cells drifting in and out of focus. Sol looked for a biological reason the cells were disappearing, while Astra recognized a z-stack, a series of images taken at different depths.

Where Astra still slips

“Higher benchmark scores do not yet mean models can make reliable judgments in the lab,” OpenAI writes. In its failure examples, Astra discounts observations that contradict its preferred interpretation. It also lets assumptions, expected answers or supplied labels override the evidence, and it sometimes makes a correct intermediate observation that then fails to guide the final decision. OpenAI added a similar caveat when Astra designed a Codex device: the model didn’t consistently account for the manufacturer’s production limits.

The clearest case is a colony PCR gel, used to check which bacterial colonies picked up the new plasmid. Astra split the colony bands into the right two groups, then assumed the larger PCR product marked successful cloning and proposed lanes 4, 6, 7 and 8. The reference answer reads the control lanes and picks colonies 5, 9 and 10. In other words, Astra grouped the colonies correctly but swapped which group was which. A lab acting on its pick would go on to purify the wrong plasmid.