A one-line fake proof passes OpenAI’s math Comparator

Comparator, the checker attached to OpenAI’s math release, accepts a one-line fake proof of the five-coloring challenge once the submitter redefines a proper coloring as False, the author of an October 9 post on Ohaithe says. The author confirmed the trick, counts six other challenges broken the same way, and says they hadn’t seen the problem discussed anywhere else.
The hole sits in the .json file that comes with a challenge. Names listed under definition_names can be freely overridden by a submission, and the author found definitions that should stay fixed in that list. When that happens, a submission can swap in whatever definition it likes and Comparator still accepts the proof. Lean, the proof language involved, has a computer check every logical step, but it checks the proof against the definitions it is given.
The author’s example is EuclideanFiveColor.lean. The whole file is one definition and one theorem. ProperColoring says a coloring of the complex plane is proper if any two points at distance 1 get different colors. no_proper_five_coloring claims that no proper coloring with five colors exists, and its proof is left as sorry, Lean’s placeholder for a missing proof. The .json lists OAI.EuclideanFiveColor.no_proper_five_coloring under theorem_names and OAI.EuclideanFiveColor.ProperColoring under definition_names.
So the author replaced the definition. The new ProperColoring is just False. Once nothing counts as a proper coloring, the theorem follows in one term, fun ⟨_, h⟩ => h, and Comparator accepted it.
The author concedes this case is easy to check by eye, because the definition sits at the top of a file that imports only Mathlib. Among the six other challenges where a similarly trivial proof gets through, the post names Bernier, LogspaceEquality (the claim that L=RL=BPL), KServer, Naimark and OccupiedOverlapRokhlinSpinAngle. The last one has a legitimate sorry inside a definition, but the author describes it as a proof obligation, not a definitional hole to fill. Comparator still lets the rest of that definition be changed, and two other definitions in its list carry no sorry at all.
The author sees no pattern in which definitions got exposed, and notes that often “only a few definitions out of a large file are given this special, broken, treatment.”
One challenge uses the field as intended, in the author’s reading. ElementaryPositivity has a definitional hole, and the author calls that correct, since its main objective is a data-carrying term instead of a Prop.
OpenAI’s openai/math README says about 42% of top-line results are formalized and points readers to the Comparator instructions for additional checking.
Up next
Claude Code Projects admits its whole Pro and Max waitlist, Video
Every Pro and Max user on the Claude Code Projects waitlist just got in, yet the waitlist itself stays open, since Projects is still a beta and newcomers are admitted only as capacity allows.
Related stories
- OpenAI’s 722 math papers come with no one to write to
- One sign error pulls three OpenAI math papers
- OpenAI’s math results cost about three hours of Pro each
- GPT-6 Astra hits 66.7% on wet-lab tasks, GPT-5.6 Sol 43.8%
- GPT-6 Astra designed a Codex device in under a week
- Reasoning off, Astra still hits 96.7% on ARC-AGI-3
Comments
No comments yet. Be the first.
Join the conversation
Sign in to leave a comment. It appears after moderation.

