OpenAI tests whether AI can think like a drug chemist
OpenAI has released MedChemBench, a test of whether AI models can reason like medicinal chemists, the scientists who design and refine drug candidates. Its 99 problems are drawn from published research. In OpenAI’s announcement on Friday, its newest model, GPT-6.2 Astra, earned 84.5% of the available credit at the xhigh reasoning setting, against 58.2% for GPT-5.6 Sol. OpenAI also states plainly that the results don’t show models can discover new drugs.
This is OpenAI’s second lab-science test this month. On BioBasicsBench, a set of routine wet-lab analyses, GPT-6 Astra earned 66.7% of the available credit.
What the test asks
Drug chemistry is hard because improving one property of a molecule can make another worse. A compound might act strongly against a disease-related protein but be poorly absorbed, or break down too quickly in the body. Chemists work through this by making and testing related compounds, then deciding what to try next. MedChemBench puts a model at a chosen point in a real research program: it gets the target protein, the researchers’ goals, the compounds studied so far and some of the lab results. It then has to rebuild the molecules from drawings, explain the evidence, rank and pick compounds, and predict their measured properties.
In one problem the model sees early results on compounds that block PRMT5, a protein studied in cancer drug discovery. It has to predict how three related compounds will behave: how strongly they block the protein in cells, how easily they cross a cell barrier, and how fast liver enzymes break them down. The later experiments, hidden from the model, showed a tradeoff. Compound 23 was the stronger blocker. Compound 24 crossed the barrier more easily and broke down more slowly.
Even the basics separate the models. Asked to assemble a molecule containing an 18-atom ring, GPT-6 Astra got it right in all ten tries. GPT-5.6 Sol and GPT-5.6 Luna failed every time. One built a 15-atom ring and the other split the molecule into two pieces, so each was analyzing a different compound.
Where the models fall short
GPT-6 Astra earned 95.4% of the credit for rebuilding structures, but only about half for choosing compounds and predicting numerical properties. OpenAI notes that these categories use different tasks and scoring rules. It also lists the test’s limits. Each answer was a single attempt without tools, feedback or new experiments, and the models may already have seen the published sources the problems come from.
Our analysis
What we checked
On November 21 we read the 27-page preprint linked from OpenAI’s post and the data behind the charts on its blog page.
The newest model has only an overall score. The preprint is dated November 9, eight days before GPT-6.2 Astra was announced, and never mentions it. On the blog page GPT-6.2 Astra appears only in the overall chart. The 95.4% and “about half” figures belong to GPT-6 Astra, and nothing OpenAI has published says how its newest model does at selecting compounds or predicting properties.
The weak spot isn’t one model’s. Across the eleven models in the preprint at xhigh, selection scores ranged from 43.58% to 52.78% and prediction scores from 37.69% to 47.02%. Structure scores ranged from 8.74% to 95.37%. The best selection score went to GPT-5.5, an older model. By our rough estimate, about three quarters of the 25-point gap between GPT-6 Astra and GPT-5.6 Sol comes from rebuilding molecules correctly, and almost none from the chemist’s actual choices.
OpenAI has also published two sets of MedChemBench scores that don’t match. Its GPT-6 Astra launch page listed GPT-6 Astra at 49.3% and GPT-5.6 Sol at 47.4%. The preprint gives 83.33% and 58.23%. Neither document explains the difference. That launch page also explains why only GPT models appear: OpenAI says it left out two Claude models because they refuse most of the questions.
Up next
Claude modeled a whole virus shell overnight, Video
Anthropic says one of its scientists gave Claude a prompt at bedtime and woke up to a way to model a whole gene therapy virus shell on one GPU, off by about an atom’s width. Would you leave Claude working alone overnight on a hard problem?
Related stories
- GPT-6 Astra hits 66.7% on wet-lab tasks, GPT-5.6 Sol 43.8%
- GPT-6 Astra designed a Codex device in under a week
- A one-line fake proof passes OpenAI’s math Comparator
- Reasoning off, Astra still hits 96.7% on ARC-AGI-3
- A harness swap took Astra from 62.7% to 99.9%
- OpenAI model planned to secretly copy emails
Comments
No comments yet. Be the first.
Join the conversation
Sign in to leave a comment. It appears after moderation.

