Skip to content
Promtime

Appearance

Appearance

OpenAI

OpenAI hands $5M to the Simons Foundation to vet AI math

6 min read

OpenAI is giving $5 million to a new Simons Foundation initiative for mathematicians working with results produced by AI. Some of the work it will fund is checking OpenAI's own recent results, which came from a model no one outside the company can run and have not been peer reviewed.

In the announcement on X, OpenAI says the goal is to help the mathematics community turn progress in AI into deeper human understanding and new discoveries. The initiative is meant to help mathematicians examine, understand and build on AI results, and OpenAI names "the results we recently published" among them.

The Simons Foundation awards the money, and OpenAI adds model access

The Simons Foundation manages the funds and decides who receives them. The money goes to workshops, conferences and special programs. OpenAI is also giving Simons Foundation-supported researchers access to its models and computing resources.

That is all the post says. OpenAI did not say how the $5 million is split between workshops, conferences and programs, when the first grants go out, or which models the researchers will get. It also does not say whether that access covers the unreleased internal model that produced the published papers.

What the funded mathematicians will be checking

On October 6, OpenAI published "Sharing AI progress in mathematics" and released the openai/math repository under the Apache-2.0 license, AICoder reports. Reports differ on its size. The README on GitHub lists 719 manuscripts in 372 families. Yahoo Tech and TechSpot both give 722.

The repository describes the process. OpenAI says it moved to open research problems after its existing mathematical evaluations saturated, meaning the models scored so high on those tests that the scores stopped telling models apart. An unreleased internal model, run with largely the same fixed procedure each time, was given approximately 4,000 problems. According to the repository, each result took on average three hours of ChatGPT Pro thinking compute. The output was grouped into families and filtered for an "appropriate level of significance".

Some results did not follow the standard procedure. The repository names two exceptions: the zero-free region for the Riemann zeta function, and the proof of the Hodge Conjecture for CM abelian varieties. The Re(s) > 11/12 writeup was edited by humans for readability. TechSpot reports OpenAI's claim that the average result needed a fraction of the compute spent on its earlier attempt at the Navier-Stokes Millennium Prize Problem, which cost millions of dollars in compute.

The catalogue may also be incomplete. TechCrunch reports OpenAI's claim that the same internal model has resolved more than 100 additional open problems across most areas of mathematics.

Only part of the work has been verified, and outside readers have found gaps

Lean is a proof assistant: a programming language whose checker accepts a proof only if every step is valid. The GitHub repository says about 42% of the top-line results have Lean formalizations, and it states that "Some of the unformalized results could have issues." Corrections are recorded as new versions. Summaries of the model's reasoning are available for only 10 results.

Problems turned up within days. On October 8 we reported that one sign error in "Algebraicity of Weil classes on split abelian eightfolds" led OpenAI to withdraw that paper and two others built on the same construction. On the same day we covered a mathematician who said Families 335 and 336 generalize his September 2 arXiv paper without citing it, and that OpenAI Support could not pass his message to the researchers responsible.

A Lean formalization does not settle a result either. On October 9 we wrote that a team led by Anders Hansen at Cambridge, according to New Scientist, found the Lean code for OpenAI's Navier-Stokes proof proves a weaker statement than the written proof: below m + 5 where the text needs below m + 4. Finding it took about two weeks. The same day we reported a write-up showing that the repository's Comparator tool accepts a one-line fake proof when the problem's definitions are redefined. It works like an exam that grades your answer but lets you rewrite the question first.

The grant follows a dispute over closed models

OpenAI says it consulted the independent Advisory Group on Mathematics and Artificial Intelligence at the Institute for Advanced Study, and that the group's advice shaped the repository's protocols for revisions and citations. OpenAI describes the group as a bridge to the mathematical community and the wider public. We covered its announcement on September 22.

TechSpot gives more of the background. The advisory effort followed an open letter, signed by more than two dozen Fields Medal winners, titled "A Severe Misalignment of AI in Mathematics". The signatories objected that proprietary models put results ahead of understanding and that researchers cannot examine the systems behind the proofs. According to the same outlet, the group gathered feedback from hundreds of researchers and then issued guidelines asking AI developers to stop testing advanced mathematical problems on proprietary models. OpenAI declined, saying it is "important to continue to evaluate our internal frontier models on mathematics and other sciences".

Mathematicians are split on the results. Alex Kontorovich of Rutgers, who responded on X the day the repository went up, wrote: "If a human did this, it would be an instant Fields Medal, no questions asked." Another commentator argued that most of the solved problems were already close to a solution and that mathematicians would soon have solved them with new tools. Others pointed to a bottleneck: humans cannot check research as fast as AI produces it.

The Simons initiative looks aimed at that bottleneck. In our view, paying people to check the results is the right thing to fund. But the access OpenAI offers is to its own models, which stay closed. That leaves the Fields medalists' central complaint, that no one outside can examine the systems behind the proofs, where it was.

The primary source is a two-paragraph post on X. The results themselves, with their version history and withdrawal notices, are in the openai/math repository. Neither says when the first awards will be made, how the $5 million will be divided, or whether funded researchers will be able to query the model that wrote the papers they are asked to check.

No comments yet. Be the first.

Join the conversation

Sign in to leave a comment. It appears after moderation.