What a Claude Watermark Does and Does Not Prove

7 min read

Anthropic has announced watermarking for Claude and says a detection API is coming. Before anyone starts making decisions on the back of its output, it is worth being precise about what such a result can and cannot support.

What a positive result indicates

A watermark test that fires on a reasonable sample indicates that the watermarked model was probably involved in producing or substantially editing this text. That is a real, useful signal, and on a long enough document the statistics behind it are strong.

It is not a statement about authorship. Consider some cases that all produce watermarked tokens:

  • A researcher writes an argument and asks the model to tighten the prose.
  • A second-language speaker writes a draft and asks for fluent phrasing.
  • Someone pastes model output and submits it unchanged.
  • Someone quotes a watermarked passage inside their own writing.

The first two are ordinary tool use. The third may be a problem depending on context. The fourth is a quotation. The detector's number is identical in kind across all of them, because it measures generation — not intent, effort or credit.

What a negative result does not prove

A miss is much weaker evidence than people assume, and there are many innocent reasons for one:

  • A different model. Output from any unwatermarked model returns nothing.
  • Timing. Text generated before watermarking rolled out carries no mark.
  • Length. Statistical tests need a sample. A paragraph may be under the floor.
  • Editing. Substantial rewriting weakens the signal, which is a documented property of these schemes rather than a defect.
  • Translation. Round-tripping through another language replaces essentially all the tokens.

So "no watermark detected" means "this specific signal was not found in this specific sample". It is not a certificate of human authorship, and no honest interface should present it as one.

Samples that are hard to judge

Three situations deserve explicit caution, and any result on them should be treated as inconclusive rather than negative.

Short text. Statistical confidence grows roughly with the square root of the token count. A few sentences cannot distinguish a watermark from chance, whatever percentage a user interface displays.

Heavily edited text. A document that has been rewritten, restructured and merged with other material carries a diluted signal. A weak result is genuinely ambiguous.

Translated text. Translation substitutes the token sequence wholesale. Expect nothing to survive.

Why "who wrote it" is the wrong question to ask a detector

Detection answers a mechanical question about generation. The questions people actually care about — did this person do the work, is this claim trustworthy, was disclosure owed — are questions about process and context. Statistics cannot settle them, and treating a percentage as if it could is how false accusations happen.

The practical alternative is unglamorous and works: drafts, version history, notes, and the ability to talk about the choices in a piece of writing. See writing with AI without misrepresenting authorship.

Reading a result responsibly

  1. Note the provider. A keyed test from Anthropic's own API is a different object from a local estimate or a style classifier. FreeAIText labels every score with its source for exactly this reason.
  2. Note the sample size and discard verdicts on short passages.
  3. Treat a positive as "possible involvement", never as "this person did not write this".
  4. Treat a negative as uninformative rather than exculpatory.
  5. Ask for the process evidence before drawing a conclusion either way.

Where FreeAIText stands today

We do not have Anthropic's key or detector, so our current score is a local research estimate of statistical watermark pressure — useful for understanding how these schemes behave, not a verdict about Claude. Anthropic's detection API has not been published; when it is, and if its terms permit this use, results here will be labelled as coming from that API and nothing else will be described that way. Current status lives on the API status page.

Sources

Keep reading