Claude learned proteins, not just papers about them

Given the start of a bacterial protein that almost nobody has studied, Claude Fable 5.2 names the next amino acid correctly 54% of the time, and only 9% of the time when the same protein is shuffled, researchers Xinming Tu and Leo Tianlai Chen report in a November 18 post on Xinmingtu. Of the 22 language models they tested, only the latest Claude models show that gap, while GPT, Gemini and seven open-weight models stay near zero, and the authors conclude that protein sequences were a sizeable part of Claude’s training.
The test is built to tell two kinds of protein skill apart. Ranking mutations of a famous protein can lean on what papers say about it: which parts sit inside the fold, which mutations cause ALS. Predicting a protein nobody has written about needs the sequences themselves. The authors say GPT-6 is good at the first kind of task and weak at the second, and the latest Claude models are good at both.
How the test works
The set is 668 bacterial and archaeal proteins from 167 families of unknown function. No family has a solved structure or more than three papers, and no other protein in UniProt, the main protein database, is 90% or more identical to any of them. Each protein is cut at five points; at each cut the model writes the next 10 amino acids, and the first one is scored. Then the same questions are asked on a shuffled copy, which keeps the same letters in a random order. A model that only knows which amino acids are common scores the same on both, so the difference measures what it knows about real sequences.
Claude Opus 5.5, with thinking at its lowest setting, is right 34.9% of the time on real proteins and 7.5% on shuffled ones, a gap of 27.4 points. Fable 5.2’s gap is 45.1 points. The gap first shows in Opus 4.7, jumps in Opus 5.5 and jumps again in Fable 5.2, and it holds in 98% of families for Fable 5.2 and 94% for Opus 5.5. Fable 5.2 also got all 10 amino acids right at 88 real cuts and Opus 5.5 at 12; neither managed it at any shuffled one.
The other labs’ models stay near zero even when they simply continue the raw sequence text through a completion endpoint, without a chat prompt.
Against models built for proteins
The natural yardstick is a protein language model, a model trained on nothing but protein sequences. On the same input, Fable 5.2’s 45.1-point gap beats ProGen3-3B at 32.5 and ProtGPT3-10B, the largest open model of its kind, at 27.7. Opus 5.5 is level with ProtGPT3-10B and below ProGen3-3B, which puts a general chatbot already on par with a 10-billion-parameter model made for this one job. ProtGPT3-10B gets more answers right outright, 38.4% against Opus 5.5’s 34.9%, but it also scores higher on shuffled proteins, 10.7% against 7.5%, because it knows which amino acids are common better.
One protein model does come closer to Fable 5.2: ESM-2 reaches a 34.6-point gap when it sees the amino acids on both sides of the gap, which the chatbots never get.
Why the authors point to training data
If the models learned from sequences, bigger protein families should be easier, and they are. On 257 families, down to 20 proteins each, Opus 5.5 improves with family size in step with ProtGPT3-10B and ProGen3-3B, while Fable 5.2 rises fastest and levels off at about 300 proteins per family. In the 128 families with no papers at all, the gaps are 45.9 points for Fable 5.2 and 26.8 for Opus 5.5, against 45.1 and 27.4 on the whole set.
Reasoning does not explain it either. The API reported thinking on 9% of Fable 5.2’s answers and 36% of Opus 5.5’s, and without thinking Fable 5.2 was right 54.7% of the time on real proteins and 8.8% on shuffled ones. GPT-6 Astra and GPT-6.1 Sol think for hundreds of tokens per question and still stay near zero.
Famous proteins tell a different story. Several models recite hundreds of amino acids of well-known proteins letter for letter, and in a first round Gemini 4 Carbon was strong on famous proteins and dropped to almost nothing on obscure ones.
That changes how earlier benchmark results read. In the PG-LLM benchmark, a bioRxiv preprint on ranking protein variants, Claude Opus 5 (Max) and GPT 5.6 Sol (Max) were the best language models, at ρ = 0.406 and 0.402, close to the protein model ESM2-650M at 0.411. Tu and Chen note that accuracy in PG-LLM improves with more thinking and does not depend on how many related sequences a protein has, both signs of leaning on the literature. On variant ranking the two labs finished almost level; on proteins nobody has written about, only Claude scores.
DNA is a different matter. Opus 5.5 predicts human and mouse protein-coding DNA but not the rest of the genome, and barely the bacterial genes behind the proteins it knows. It wrote all 20 next letters correctly in 25 of 500 human coding stretches; a blind guess does that about once in a trillion.
Filters and access
The authors ran the Fable models through OpenRouter, because Anthropic’s API serves them only to workspaces that retain data. Anthropic’s usage-policy filter blocked 44% of Claude Fable 5’s questions as cyber content, against 2.5% for Fable 5.1 and none for Fable 5.2, and Claude Sonnet 4.5 refused 97% of requests that started its reply with a protein sequence. OpenAI rejected 6.6% of GPT-6.2 Astra’s main-set questions as possible biological misuse, and Gemini 4 Carbon blocked about 75% of DNA prompts outright. Chat models also need the instruction: sent a sequence alone, Opus 5.5 almost always describes it instead of continuing it.
All code, test sets and model outputs are in llm-bioseq-probe; the run cost about $730 in API fees. The authors’ next question builds on how protein models are used: ESM-2’s internals hold enough structure for a small add-on, ESMFold, to predict 3D shapes. “Could Claude fold a protein, or rank its variants, the same way?”
Up next
Claude gets write access to Microsoft 365 on Dec 9
Claude’s Microsoft 365 tools for sending work email and Teams messages switch on December 9, Claude’s help page says, set to ask first unless an admin blocks them. Would you let Claude send your work email, or keep it read-only?
Related stories
- Claude mapped the first prokaryotic vault structure
- Scorpion toxin binders: 17 of Claude’s 211 designs worked
- Claude Fable 5.2 hits 69% on PCB layout test, up from 25%
- METR clocks Anthropic benchmark gains at 1.9x old pace
- Sonnet 5 aligned an early Opus 4.8 checkpoint
- Claude Fable 5.1 cracks Urquhart’s Cyphral Distich
Comments
No comments yet. Be the first.
Join the conversation
Sign in to leave a comment. It appears after moderation.

