OpenAI denies anyone looked at user data for Navier-Stokes

OpenAI answered the mathematicians who say it learned from their ChatGPT conversations with a two-part statement on September 12. No human or agent looked at user data during its Navier-Stokes work, it said, but user feedback and de-identified data do go into improving ChatGPT and Codex. “And so does every LLM company,” the reply posted on Twitter adds.

The two halves answer two different questions, and the critics have been asking both. Andreas Thom, a geometry professor at TU Dresden, kept them apart in an email to OpenAI’s Mark Sellke and Sébastien Bubeck, as heise online and OfficeChai both reported. He wrote after OpenAI announced in early August that its Astra model had constructed the first non-sofic group, settling a 27-year-old open problem in geometric group theory first posed by Mikhail Gromov. The proof’s central technical step leaned on a 2019 paper by Gábor Kun and Thom and on a related 2016 result by Kun. Thom says he and a Dresden colleague had spent months discussing the expander matching problem and extensions of the Kun-Thom work with ChatGPT. So he asked two things: whether those conversations had entered training data, and whether the system could access them while it worked on the proof.

Thom quotes Sellke’s answer as “Regarding your conversations with ChatGPT: that did not happen.” In Thom’s view it addresses only the second question, and he calls it “unjustifiably broad and materially misleading.”

The September 12 statement is about a separate dispute. Per OfficeChai, NYU’s Tristan Buckmaster and Anthropic researcher Levent Alpöge accused OpenAI of learning about their unpublished work and rushing out a competing Navier-Stokes proof. OpenAI’s earlier written response to them said no researcher or agent saw the pair’s work before publication and no specific user data was accessed. It added that it “cannot rule out that de-identified data derived from their usage of our products helped improve” its models. The new reply keeps that same split, only more bluntly: a no on access during the work, and a yes on training in general.

De-identified data is user text with the details that tie it to a person removed. On its own pages, OpenAI says ChatGPT improves by further training on the conversations people have with it unless they opt out. It also says it has built technologies to help models learn useful general patterns rather than private information about individuals. Thom’s objection, as heise online reports it, is that de-identification can remove a name but not the intellectual content of a mathematical idea. With an unpublished line of attack on an open problem, the general pattern and the private information may well be the same thing.

Thom says he turned off training on his content on June 29. He points out that this control only works going forward and that users cannot audit it from outside. It also leaves open what happened to his earlier conversations, or to data already derived from them.

He also found it suspicious that Astra took the Kun-Thom route, which by his account was not considered the most obvious path to a solution. OfficeChai says the approach was never the leading candidate for resolving non-soficity.

What Thom is asking for is a disclosed basis behind any broad denial: which data, which model versions and which account settings it rests on. Per OfficeChai, that means OpenAI’s training checkpoints and a plain-English definition of “de-identified data derived from usage.”