Skip to content

OpenAI

OpenAI’s text watermark fades after a 25% synonym swap

4 min read

Written by an AI model from the linked sources and checked against them by a second model. How we work

All about: OpenAI models and products →
Image: openai.com

OpenAI on October 5 let API customers worldwide opt in to an invisible text watermark for select models. It also said eligible ChatGPT and Codex output in the European Union will carry the watermark within weeks, on every plan, as its answer to the EU AI Act’s rule that generated text be identifiable by machine. API text stays unmarked unless a customer turns the watermark on, and the detector goes only to researchers and expert organizations that OpenAI approves, so nobody outside that group can check a passage for the mark yet.

For API customers the watermark is off by default. OpenAI presents the opt-in as a way for customers to decide how watermarking fits their transparency obligations and the experiences they give their users. It is also working with cloud partners to bring watermarking to OpenAI model outputs served through their platforms in the coming weeks.

ChatGPT and Codex get it the other way round. The watermark goes on eligible output for users on all plans, in the EU only. OpenAI says it is not making text watermarking a global default at launch, and that the regional rollout gives it room to learn from real-world use and feedback.

Applications for the detector opened the same day. Following the Code of Practice, access is granted case by case. The tool reports whether it finds an OpenAI watermark, without identifying the user or revealing prompts or conversations. The restriction covers text only: the image and audio checks at openai.com/verify and through the Content Provenance API stay open to any organization.

How textGrain marks text

textGrain, as OpenAI calls the technology, adds an invisible statistical signal to the model’s word choices, and the detector looks for that signal in a passage. The mark lives in which words the model picks. The invisible characters people find in pasted text are a different mechanism: they are often formatting residue from word processors, chat interfaces and web templates, and they turn up in purely human documents too. Stripping them leaves the words, and so the watermark, untouched.

OpenAI says textGrain matched or exceeded the other approaches it tested, SynthID for text among them, and adds that strong performance under ideal conditions does not guarantee reliable detection in everyday use. A technical report describes the method and will get more detail in the coming weeks. OpenAI also plans to release the technology as open source.

Where detection breaks

A detector can fail two ways: it can report a watermark in text that has none (a false positive) or miss one that is there (a false negative). With the false positive rate targeted at 1%, OpenAI’s detector found the watermark in about 80% of 200-token passages and about 95% of 400-token passages on content such as psychology, using watermarked answers to questions from the ELI5 dataset. On mathematics, where there is less freedom in word choice, detection was substantially lower.

Editing does more damage. In a test on 400-token watermarked English answers to ELI5 questions, swapping 10% of the words for synonyms already cut detection from about 92% to 66%. Swapping 25% left it at 17%. OpenAI says these limits are part of why the detector goes first to approved researchers and expert organizations, who can help it judge reliability and responsible use.

Does the mark cost quality?

On the benchmarks it uses to assess Astra, its latest frontier model and the top of the GPT-6 family for the hardest reasoning, OpenAI reports no meaningful difference with the watermark on. All the figures are OpenAI’s own runs, with Astra at max effort.

  • Watermarked text scored higher on five of the eight benchmarks.
  • Terminal-Bench Science 0.1 showed the largest change of the eight, rising from 56.90% to 60.00% with the watermark.
  • DeepSWE v1.1 had the largest drop, from 72.80% to 71.68%.
  • GPQA Diamond went from 94.44% to 93.94%, and the Artificial Analysis Intelligence Index from 49.57 to 49.76 points.

What a match proves

OpenAI says a watermark can show that one of its systems generated or processed part of a passage. It does not measure how much human judgment or editing went in. It does not establish who owns the text, whether its use was lawful or who is responsible for it. It does not tie the text to a person, account, prompt or conversation, and it says nothing about whether the passage is accurate.

A negative result tells you even less. Text made with OpenAI tools may be too short, edited or translated for detection to work, may come from an unsupported model or predate watermarking, and text from another company’s tools carries no OpenAI mark at all. “The absence of a detected watermark does not prove human authorship,” OpenAI writes.

No comments yet. Be the first.

Join the conversation

Sign in to leave a comment. It appears after moderation.