Skip to content

OpenAI

OpenAI shelved a text watermark in 2024. EU fines forced one

5 min read

Written by an AI model from the linked sources and checked against them by a second model. How we work

All about: OpenAI models and products →
Image: businessmodelanalyst.com

OpenAI is turning on textGrain, a statistical watermark for ChatGPT and Codex text, by default in the European Union, and leaving it off everywhere else unless a developer opts in through the API, Rews reported on October 10. If your team publishes AI-generated text in the EU on matters of public interest, the embedded mark does not cover your own labelling duty under the AI Act. OpenAI’s own report also shows detection on 400-token passages falling from about 92% to 17% once a quarter of the words are swapped for synonyms.

The law splits the job in two. Article 50(2) requires providers of generative systems to mark synthetic audio, image, video and text so that machines can detect it. Article 50(4) separately requires deployers who publish AI-generated text to inform the public on matters of public interest to label it as such, and a provider’s mark under 50(2) does not by itself discharge that obligation. The transparency rules took effect on August 2, and already-deployed systems have a transition period running to December 2. Breaching Article 50 can cost up to €15 million or 3% of a company’s global annual turnover, whichever is higher, under Article 99.

Scope depends on the product surface more than on geography. ChatGPT and Codex output is marked in the EU. The API stays off by default worldwide, and outside the EU textGrain is an opt-in switch for select models.

What OpenAI’s figures show

At a 1% false-positive rate, OpenAI’s detector finds its watermark in about 80% of 200-token ChatGPT outputs and about 95% of 400-token ones, according to the company’s technical report. Light editing changes that fast. Replacing one word in ten with a synonym takes detection on a 400-token passage from a reported baseline near 92% to 66%; replacing one in four leaves 17%.

Length matters because the test gains statistical power with every token. A two-sentence reply, a headline or a single paragraph lifted out of a longer piece sits where the signal is weak before anyone edits a word.

OpenAI attaches its own caveats, as Neowin lays them out: strong performance under ideal conditions “does not guarantee reliable detection in everyday use,” a missing watermark proves nothing about human authorship, and detection cannot show how much a user edited the text.

How the mark gets into the text

textGrain adds no hidden characters, unusual punctuation, invisible spaces or extra tokens. At each token it reweights the plausible next words using a secret key and random noise, so the model leans toward whichever candidate correlates with that key. An “entropy budget” caps how much variety the model may give up per token. That cap is the basis for OpenAI calling the scheme “unbiased”: averaged over many generations and many keys, the token distribution matches unwatermarked output, even though any single response has been steered. The detector holds the same key and runs a statistical test over the token sequence to see whether the word choices fit the keyed bias better than chance.

OpenAI wrote the method up with researchers at Yale and the University of Pennsylvania. Its benchmark comparisons have textGrain matching or beating Google DeepMind’s SynthID-Text on the metrics it tested, a self-reported result with no independent replication yet. That comparison point has its own record: researchers at ETH Zurich’s Secure, Reliable, and Intelligent Systems Lab found SynthID-Text easier to scrub than other state-of-the-art watermarks, even for an unsophisticated attacker. Google’s own Nature paper from October 2024 remains the only one of the three schemes with large-scale field data, measuring quality across nearly 20 million live Gemini responses with no detected degradation.

Who can run the detector

For now, vetted researchers only. OpenAI’s help pages name three initial recipients: John Thickstun at Cornell, Martin Vechev at ETH Zurich and INSAIT, and researchers at the Kempelen Institute of Intelligent Technologies, with a request process for others. The Commission’s Code of Practice, finalized in July and signed by roughly 190 companies by the end of that month, OpenAI included, is what lets providers limit detector access this way.

Outside tools are left without an official way in. In August, the open-source WMTrace workbench launched with a rule against shipping any OpenAI or Anthropic detector until an official verification API or a reproducible public specification exists. OpenAI says it intends to open-source textGrain eventually; no open version exists yet, so there are no robustness results for one.

Anthropic made a different call on scope. Its watermark, announced in August and adapted from SynthID-Text, applies to Claude everywhere. “We’re applying watermarking globally at launch because we don’t yet have a durable way to scope it by region,” the company said. Its detection API is in private preview for regulators, law enforcement, media and fact-checking organizations, and researchers.

Shelved in 2024

In August 2024, TechCrunch reported that OpenAI was sitting on a ChatGPT text watermark, one the Wall Street Journal said was 99.9% effective on sufficiently long passages. OpenAI said at the time it was taking “a deliberate approach.”

Internal surveys found that 69% of ChatGPT users thought such a tool would lead to false accusations of AI-assisted cheating, and 30% said they would use ChatGPT less, or move to a rival without a watermark, if it shipped. OpenAI’s spokesperson called the approach “trivial to circumvention by bad actors” through translation or rewording with another model, and flagged a risk of disproportionately flagging non-native English speakers. The synonym-swap figures in the textGrain report measure that same rewording weakness.

The difference this time appears to be the regulator’s fine. OpenAI’s announcement sets the limit in one line: “We are not making text watermarking a global default at launch.”

No comments yet. Be the first.

Join the conversation

Sign in to leave a comment. It appears after moderation.