Tübingen researchers want AI to propose many models

AI agents that build scientific models should return a weighted set of candidate models instead of one best fit, Jakob Macke, Daniel Gedon and Stefan Wahl argue in a guest post on Anthropic’s Science Blog published November 6. A single model from such an agent is one of possibly several that explain the data equally well: run the agent twice on the same data and it may return two different ones.

The three work at the ELLIS Institute Tübingen and the University of Tübingen, where they build AI tools for developing scientific models in neuroscience and other fields. They are grantees of Anthropic’s AI for Science program, which gives free API credits to researchers working on scientific projects.

What the agents do, and what the authors change

The setup they describe is already common. An LLM-based agent gets some data and a natural-language description of a system, writes code for one or more candidate models, runs them, fits their parameters, compares the output with real measurements and revises, for instance by changing the form of an equation. The loop repeats until the model reproduces the data closely enough. Several such systems have appeared in the past two years, and they all use the LLM to design the model itself, a step the authors say was until recently thought to be something only humans could do.

Their objection is that reproducing data proves little. Many models with differently shaped equations can fit the same measurements, and an agent that reports only one of them hides the rest.

So the authors want the agent to search the space of models. It proposes many candidates, and each one is weighted by how well the data supports it. Models that need a lot of flexibility to fit get less credit than those that fit naturally. Weak candidates drop out, and the agent proposes variations on the stronger ones. What comes back is a distribution of plausible models. If every survivor shares a feature, the data likely favors that feature. Where the survivors disagree, the evidence is thin.

Borrowed from parameter fitting

The idea moves up a level something scientists already do with parameters. Within a single model, different combinations of parameter values can produce near-identical output: in nerve-cell models, different gate settings yield almost the same electrical activity. Bayesian inference handles this by returning every parameter set that fits reasonably well as a probability distribution. But it requires calculating how likely the measurements are under every possible set of values, which becomes computationally impossible for anything much more complex than the classic nerve-cell model.

Simulation-based inference (SBI) gets around that with brute force. You draw random parameters, run the model and record the output, perhaps 100,000 times for the Hodgkin-Huxley nerve model and millions of times for larger ones. Then a neural network trained on those runs learns to return, for a given output, the range of parameters that could have produced it. The only requirement is a simulator, any model you can run with parameters in and data out, including ones that can’t be written as a tidy equation.

This half of the program already works outside the lab notebook. The simulations are expensive and can take weeks, but the trained network then performs inference on new data in seconds, with no further simulation runs. When two neutron stars collide, SBI can locate the collision on the sky roughly, fast enough to point telescopes at it while the signal is still detectable. The authors list cosmology, structural biology and geophysics among the fields using it. And because the output is a distribution, it shows which parameters the data pins down, where more than one explanation fits, and where another experiment would help most.

Why one equation isn’t enough

The authors’ case rests on the model they open with. In 1951 Andrew Huxley and Alan Hodgkin wrote equations for how a squid nerve cell fires. Cambridge University’s computer was out of action for six months, so Huxley solved them on a hand-cranked mechanical calculator, which took him over three weeks.

To make the model fit their measurements, they had to raise one variable to the fourth power without knowing why. They tentatively suggested that several charged particles had to move into place before a membrane gate would open. Decades later, after the pair’s 1963 Nobel Prize, another team found the structure of a related potassium channel protein: four identical pieces around a central pore. The model had predicted a structure it was never fit to.

The authors say an AI system could plausibly find an equation like that in an afternoon, perhaps with an even closer fit. What made the 1951 result useful was knowing that simpler versions of the model did not work. A list of surviving and rejected candidates gives that comparison, and a single answer does not. In the authors’ words: “An AI system that merely hands you the model might give you an answer, but it does not give you understanding.”