Anthropic’s Claude watermark follows SynthID

Anthropic is now watermarking Claude’s outputs with a scheme that anyone will eventually be able to check. The scheme is based on Google’s SynthID, and SynthID in turn builds on a method Scott Aaronson designed at OpenAI in 2022, one OpenAI then chose not to ship. Aaronson laid out that history in an August 22 post on his Shtetl-Optimized blog. He calls his Gumbel Softmax scheme, as far as he knows, the first proposal for watermarking LLM output, and says Anthropic credits him for it even though he never published a paper. By the time he sat down to write one, he says, “it seemed like the whole field had already assimilated my scheme and moved beyond it.”

The watermark adds nothing visible to the text. An LLM already samples each token at random from a probability distribution. The scheme swaps part of that randomness for pseudorandomness from a generator with a secret key, and the swap quietly favours certain word combinations. To detect it you need only the token sequence and the key. You don’t need the prompt or the model’s probabilities. Given those, a statistical test can show with high confidence that a text came from that specific model. Aaronson notes that Christ, Gunn and Zamir later improved his scheme to reach true cryptographic indistinguishability, so watermarked output can’t be told apart from unwatermarked output without the key, and that more improvements have followed.

Why the text doesn’t get worse

The usual objection is that steering the sampler must cost quality. Aaronson argues that it doesn’t, and he points to Zvi Mowshowitz’s post “AI Text Watermarking Is Free And Good” for the full case. “Basically, nearly every LLM output was already a sample from a cloud of exponentially many possibilities, all of them about equally good,” Aaronson writes. That leaves room to steer inside the cloud without changing anything an ordinary user would notice.

The original design did draw one criticism on output. The authors of the GumbelSoft paper credit the Gumbel-max watermark to Aaronson and Kirchner in 2023 and describe it as known for exceptional detectability and low perplexity, which means the marked text still reads as fluent. But they say it always returns identical outputs for the same prompt, because both the pseudorandom function and the decoder are deterministic. In their view this hurts generation diversity and user experience. Anthropic’s version comes by way of SynthID, which Aaronson calls “very similar” to his proposal, not identical.

Getting rid of it

The weak point is removal. Aaronson admits that schemes like his and Anthropic’s can be stripped with a little extra work. His examples: translate the text from English to French and back, ask the model for words mixed with emojis and then delete the emojis, or run the output through an open model to paraphrase it. He had already named this an unsolved problem in an August 17, 2023 talk at the Simons Institute: how to build a text watermark that survives translation, paraphrasing and similar attacks. Three years later, Anthropic is shipping a scheme that is still open to them.

Paraphrasing is less all-or-nothing than it sounds. In August, tests run with the declaude rewriting tool found that a full rewrite left about 0.5% of windows intact, while lighter edits did far less damage. Mowshowitz, Aaronson says, gives detailed reasons to expect watermarking to stay a net positive anyway. Aaronson adds recent progress on what he calls “semantic watermarking,” which marks the underlying concept vectors instead of the tokens. It “actually seems to work,” he writes, though with no theoretical guarantees, and it should make removal much harder. He also cites a result from Barak et al. which suggests that, under plausible assumptions, no LLM watermarking method will ever be completely foolproof.

Four years to deployment

Aaronson worked out the scheme in fall 2022 and gave many talks on it, including one at Anthropic. At OpenAI, Hendrik Kirchner implemented and tested it. OpenAI’s leadership then decided not to deploy it. Their main worry, Aaronson says, was product risk: customers might dislike the idea and move to a competing LLM with no watermark. He refers readers to a Wall Street Journal investigation from two years ago for more on that decision. He had hoped California would solve the collective-action problem with a mandate, but the state limited its requirement to audiovisual content and left text out.

Google DeepMind went further and put SynthID into all of its Gemini text models. But it heavily restricted who can run detection. That, Aaronson says, made the watermark of limited use to the academics who keep asking him for a way to catch students who cheat with AI. For now he sends them to Pangram, an AI detector that doesn’t rely on watermarks, as a first line of defence.

Anthropic is taking the opposite approach on detection: anyone will be able to check Claude text. Aaronson says the move appears to be for compliance with EU regulations. Anthropic’s own FAQ says the company is still working on the detection API, so the open check hasn’t arrived yet. Aaronson adds that OpenAI suggests it plans to follow suit.

“So, four years after I seriously thought about this, it looks to my surprise like this is actually happening,” Aaronson writes.