WMTrace won't ship an Anthropic watermark detector

WMTrace, an open-source forensic workbench for LLM text watermarks published on Github under Apache-2.0, ships with five detectors and a standing rule that no anthropic or openai detector will exist in it until an official verification API or a reproducible public specification does.

At a glance

• The detector set covers a KGW-style contextual green-list with γ=0.25 and a 2-word context, a fixed keyed green-list, zero-width Unicode payloads, synonym-pair bits, and a keyless Unicode inspector.
• Output arrives as versioned evidence bundles graded E0 to E5 rather than a percentage score, and the bundled demo sample reports z = 16.1 with a Holm-adjusted p of 3.5e-48.
• No detector in this build can reach E4, the grade that requires trusted key provenance, so every result stops short of binding a piece of text to a named provider.

Why it matters The design reads as a deliberate limit on what a detector may claim: four problems kept in separate namespaces in code, API and interface, abstention states that fire before scoring, and a top grade this build cannot reach. For teams handling disputed text, the practical effect appears to be a narrower question answered honestly, with the scope statement attached to every bundle.

Two keyed detectors reach grade E3 and three stop at E2

kgw-toy and unigram-toy are keyed against a published demo key and can reach grade E3, the level reserved for calibrated evidence of a declared scheme, key and configuration. The zero-width and lexical-codebook detectors are unkeyed and cap at E2, as does the keyless unicode-inspector.

Four of the five carry an [ILLUSTRATIVE] claim status; the Unicode inspector is marked [IMPLEMENTATION]. Below the scoring grades sit two abstention states: E0 for unsupported input where a required key, tokenizer or configuration is missing, and E1 for compatible text carrying too little usable signal.

The project, posted to Show HN, documents its limits directly: a negative watermark result does not mean text was human-written, and a recovered payload does not prove who embedded it. No result supports provider attribution without trusted key provenance, and all keys in the build are published research keys.

A shared statistics layer owns the exact binomial test, z and Holm correction

Each detector implements a five-method plugin contract: manifest, capabilities, score, calibrate and explain. Significance testing lives in a shared statistics layer, so no individual detector rolls its own, and normalization is never silent, since every bundle names the representation it analyzed, raw, NFC or NFKC.

Input passes a hash and representation step, then a capability check that can return E0 or E1 before any scoring, then the detector plugins, the statistics layer, and finally evidence bundles emitted as JSON, terminal output or web cards. The suite runs 41 tests, with the golden set reproducing every worked example in the research spec.

On the command line, wmtrace scan runs all detectors with Holm correction across the run, wmtrace detect targets one declared scheme, and wmtrace inspect limits the pass to Unicode and formatting. Embedding is reproducible: wmtrace embed --scheme kgw-toy --length 150 --seed 11 writes a sample.

The demo sample scores z = 16.1 with a Holm-adjusted p of 3.5e-48

The Analyze tab returns one evidence card per detector: grade badge, decision, sample size, green count, the z statistic, exact binomial p, Holm-adjusted p, a per-token green heatmap, Unicode findings and a scope statement. On the bundled watermarked sample it reports z = 16.1 and a Holm-adjusted p of 3.5e-48.

The Embed tab generates green-biased sample text under the demo key or hides zero-width and lexical payloads in user-supplied text, with a one-click round trip back into the analyzer and invisible characters revealed inline. The About tab holds the evidence-grade table and detector manifests.

The interface runs at 127.0.0.1:8177 after an editable install on Python 3.10 or newer, and is described as responsive, mobile-friendly and switchable between dark and light. A ?demo=1 query parameter auto-loads a watermarked sample and analyzes it; ?theme=light or ?theme=dark forces a theme.

What's next The source lists a SynthID-Text reference implementation, Fast-DetectGPT, Binoculars, C2PA verification and provider adapters as roadmap items, with the full plan and reasoning in the research spec, which sets out a P0 to P6 implementation sequence. Whether any provider-bound detector follows depends on an official verification API or a reproducible public specification appearing, the condition the project has set for itself.