Mainsprings, watermarks, answer boxes and other bad words for good ideas

Lotta discourse the past few days about Anthropic’s decision to watermark Claude content. Not a lot of correct discourse on what it’s doing or why. My hunch for why lies in the decision to call it a watermark, rather than the process of watermarking at all.

For context, I’m a developer advocate at Pinecone, a vector database company. A lot of my job is figuring out clever metaphors, heuristics, and mental models for people to understand what vector databases are for and how they work.

Importantly, a rule I follow when writing content is never assume what the reader know. Specifically, I am very careful in introducing terminology that can have existing meanings or predictive power.

I find that people form a lot of quick impressions of how things work, and I think the watermarking fiasco is particularly indicative of this.

In order to demonstrate this, I’ll write about a few mental models that have mangled my and others understanding of how things work, and how I think this explains the rage expressed at Anthropic.

Watch this

Photo by Kristoferb, public domain.

A few weeks ago, I went to the Chicago Windup Watch fair with my buddies, and I bought a particular watch that caught my eye.

I’m not a huge watch guy, but I got gifted a Brew Metric Lite watch from my fiancee, and I fell in love!

If you are familiar with watches, you may know that they use many different mechanisms for power. How watches work exactly is beyond me, but my understanding is there’s a bunch of gears that move in precise ways to represent the silly arbitrary units of information we call time. And all those tiny gears need power to move and that’s where these energy sources come in.

There are battery powered watches which use chemical energy for power, and watches that store potential energy in the form of various springs.Of those watches, there are automatics, which are “self-winding”, which I am the most familiar with. Simply wear these watches, and they leverage a weighted spinny thingy inside the watch to wind a spring to charge your watch.

Photo by Petar Milošević, CC BY-SA 4.0.

There are solar powered watches, notably the Citizen Eco-Drive series, which charge based on literally any light source. Again, some sort of chemical black magic invented by really smart people. And, there are even static powered watches, which some how pick up the static energy stored in your body and arm hairs to self charge.

The watch I ended up picking up is a wind-up watch from Studio Underd0g, which means it does not self-wind like an automatic.Specifically, you have to rotate a dial on the side of the watch to wind it up.

I bought the watch thinking it was an automatic (as I saw a window to the mechanism on the back and just assumed that it would be an automatic, failing to notice the lack of a weighted mechanism to wind itself), and now had the question of how actually I should take care of this watch.

So, I opened up the manual and tried to learn.

The instructions were very simple, roughly as follows:

• This is a wind up watch
• It takes roughly 20 rotations to wind and lasts about a day or two
• You rotate the crown clockwise until you feel resistance
• Do not overwind. Overwinding may break the watch

As you can imagine, I was quite alarmed by the prospect of bricking my watch I bought, and wanted to learn more.

I took to the website watched a video about how to wind thw watch.. Fortunately the company has an awesome video on what to do, which seemed to help. I highly recommend you watch video to understand some points I’ll make later, but it’s short and has the following quotes that stuck out to me:

“Mechanical watches don’t use batteries to power their movements. They use potential energy stored in what’s called a mainspring for their movements. In order to charge the mainspring, you need to wind the movement.”

And then, after demonstrating how to wind:

“You can carry on winding until you start to feel resistance from the crown. At this point, your watch is fully wound”

After watching the video and feeling the watch in my hand, I felt the watch worked as follows:

• there was some sort of spring called a mainspring inside that is being wound as I turned the dial
• At some point, the winding force the spring pushes back on me will be too great and offer feedback to stop
• I stop. Further winding would cause catastrophic damage internally

At the end of my first day wearing my watch, I realized that it kept losing time. I was so worried!

Did I have to return my watch, which could take who knows how long?

Am I just out a significant sum of money I could have spent on literally anything else?

My constant worrying lead into the next thing I did which was Google, and by Google I mean Reddit and I found my answer deep in a watch enthusiast thread.

The answer came from another redditor in a similar position. They asked a company rep what was going on, and I’ll reprise a bit of the answer here.

The mainspring has a “two-step” resistance behavior. You will feel significant resistance at some point during winding, but this is the process of winding and not the end. Keep going until absolute significant resistance is reached.

In other words, the spring really worked like this:

1. Wind the watch
2. Feel step 1 of resistance, keep winding
3. Feel significant resistance at step 2, which is the stopping point
4. Wind until you can’t anymore reasonably

And there’s more. The mainspring (which in my head was a tiny spring that was maybe the most large tiny spring in the watch, kinda like a pen spring) is actually not a long spring but a flat winded spring, which wraps around the whole watch. And, the mainspring comes with this two-stage mechanism specifically to prevent easy breakage.

Mainspring of a Sea-Gull ST16 automatic watch movement. Photo by Hustvedt, CC BY-SA 3.0.

After adopting this model, I’ve been able to wind the spring without damaging really anything!

So, why did I spend so much time talking about watches? Well, the types of development concerns, issues and self-conversations that can occur during software development and developer tool evaluation, is quite similar to when people choose everyday things and products.

In fact, try to go back and look at my overall journey:

1. I had a problem (I was at a watch fair and wanted to buy a watch)
2. I found a solution (a wind up watch)
3. I applied my existing understanding of what watches were (automatics) to my current watch, incorrectly
4. I made assumptions, tested them, failed (looked for a video and learned stuff)
5. I made the wrong assumptions from the video again (more learning-by-trying)
6. I looked on reddit, and found the right answer (ding!)

I’d be willing to bet that developers in general take a really common path to understand devtools as well! With maybe a Claude chat or two in the middle.

Let‘s look at another instance, this time closer to tech and AI.

Broken answer boxes

If you’ve found this article via my newsletter Answering Machines, you’ll realize that that is my metaphor for how large language models work. And at my job at Pinecone, I often have to help people understand how to make language models work with their own datasets, for a retrieval architecture called RAG (retrieval augmented generation).

As a quick refresher, retrieval-augmented generation or RAG is a design pattern for building AI applications that work with databases, usually unstructured or document data. The idea is that agents don’t know about the information at your company, so giving them the ability to find that information and then generate an answer is more reliable than asking agents directly. This design pattern is the foundation of probably all modern AI systems that require using up-to-date information.

A diagram on RAG from the Pinecone blog, demonstrating a complex pipeline for customer support

I would also argue that misunderstanding how complicated each word in the RAG acronym works describes a majority of problems developers face.

Developers want RAG when they want to avoid hallucination, add up-to-date information to their models, or to offer a simple form of traceability to when answers occur. RAG is also useful for providing a natural language interface over a knowledge base, and some particularly intelligent LLMs can reason together information from disparate documents retrieved to handle complex queries.

I’ve found in my work at Pinecone that many new practitioners for applied AI adopt a model of RAG as creating an “answer box”.

To be clear, this isn’t something that developers talk about, but rather, a set of wrong predictions I think developers make about how their systems should work, due to the implicit model they have of it.

In my description, an answering machine is an LLM, and an answer box is any database that contains potential answers. RAG then, is answering machine + answer box. Confused yet?

Tortured metaphors aside, the point I’m making when developers run into issues developing RAG is that they think that the mere presence of the answer in the box will require the system to answer correctly. In other words, a phenomena I have tried and failed to coin as:

If answer in box, why not box answer?

This failure point breaks the brains of developers; how can it even be possible for a RAG system to answer incorrectly, if you ask an obvious question and need to find an answer?

This failure mode isn’t as obvious as it seems either, it can cause developers to make the following assumptions which are false:

• passing in a keyword-heavy query will succeed with a semantic-search only query
• Passing in a query that is inherently multi-faceted, but requiring only single documents to be returned
• Passing in questions which require chunks larger than those used to answer

And even more. The issue is that this metaphor, which implies a world where you throw your documents into a database, and then attach the vector database (through any means) to your LLM, implies that the connection means availability. But it doesn’t! It means it is liable to be retrieved but not guaranteed. And this metaphor, of searching versus availability, is the major failing that a box-like description of databases can cause.

Great, now we know that having the wrong words for complex things can cause a lot of confusion. So what’s going on with watermarking?

Why watermarking confused developers

Suppose you are tasked with determining if a given piece of text is AI generated or not. You might consider this challenge laughable in 2026 (for reasons we’ll get into) but suspend your disbelief for a moment.

You have a few options available:

1. you could train a model on human/AI generated text, and predict classification scores.You’d need to train this model constantly to keep up with model updates

2. You could enforce some sort of mechanism that cleanly self-identifies what LLMs generate what across the ecosystem. This would require coordination and standards adoption, as well as some sort of inlay into open source implementations

Now why would you want to do this? Turns out, it’s really useful to understand the likelihood if a given text is AI generated. For some reason people dislike this…..

As AI generated content spreads, and models get better, model labs and regulators such as the EU are realizing the potential dangers of not having an accurate system to detect such material. And so, a few companies adopted option 2 and decided to implement a technique called watermarking to accomplish it.

Now, when you hear about watermarking, what do you think of? You might have heard about watermarking from seeing Getty images or the like with an obvious stamp on designating the origin of a photo.

A visible watermark, the kind most people picture. Photo by Barry Taylor, CC BY-SA 4.0.

Or, you’ve maybe thought of checks or other important government documents that have stamps or security-related insignias designating their general importance and origin.

What’s common with all of these artifacts? They are blatantly visible and easy to identify with the human eye, or at least with some trivial manipulation.

So when Anthropic released their watermarking capability recently, people, frankly, lost their minds. The intention for Anthropic was of course to attempt to solve the strong negative sentiment that AI generated content has alongside regulatory concerns.

There were a lot of different opinions, some more legitimate than others.

What I’ve heard ranges from the reasonable (“”Does this degrade output quality”) to the insane (“Anthropic is claiming YOUR ideas as its own!”). What’s interesting to me, however, is why the opinions vary at all.

A lot of these takes only make sense if you think watermarks work a certain way. And once you outline how they work, it becomes indefensible.

Because, as it turns out these watermarks are not the same as the other watermarks we listed!.

These are a special kind of watermarking process that is more akin to a digital watermark, than a physical one.

You see, digital watermarks bear no requirement to be perceptible to the human eye at all, unlike their physical compatriots. And the way Anthropic’s watermark is (supposed) to work is even more clever: they modify the distribution of tokens selected for each text generated in such a manner, that after a certain length of text, the trajectory of tokens selected is clearly indicative of coming from an Anthropic model.

Most importantly, the watermark itself is not perceptible semantically at all either. If all goes well, the generated text should have the same meaning as a non-watermarked text; the watermarking process does not meaningful change the quality of the output.

The best mental model I have for this is the following:

Imagine you are staring at a map. And at certain parts of the map, there are places marked “Anthropic space” and “open source model space” and so forth. You start from the left side of the map, and as you “generate” text, you move from one spot to another, tracing out a path. Please note that this too, is a poor mental model, but a bit more tactile on the process of the watermarking.

If watermarking was not applied, you’d take some trajectory from “no devs space” to Anthropic space. If watermarking was applied, maybe you’d have a tiny random nudge each time you choose a distance to hop, so tiny you wouldn’t notice it comparing to the previous nudge, but enough nudges that over your long journey tracing the path, you’d clearly end up in “watermarked Anthropic space”.

If you want a better technical explanation, I really like the flowchart from the SynthID-Text paper that Anthropic references:

Fig. 2 from Dathathri et al., 'Scalable watermarking for identifying large language model outputs,' Nature (2024), CC BY 4.0.

Although I should note that Anthropic’s own blog post on the matter is not bad!. It probably should have included some visuals to explain to developers how it works though.

This is not, perceptibly modifying the output text so it says “Anthropic wrote this” every six lines. And the only way to do so, is to understand what the model would have generated in the first place, and the modification rule or schema or algorithm, which a normal person cannot do. Since the watermark is actually digital and semantic, you theoretically should not be able to sniff it out without access to the codebook/generating recipe.

Funny enough, the first time I encountered this idea was back in 2023, when I worked my last job tasked with building AI generated content detection systems. The research paper that defined watermarking technique had just come out and it was pretty awesome and easy to read. I highly recommend reading that directly, or Sebastian Raschka’s excellent visual breakdown on how the algorithm works, or even the followup paper that Anthropic ended up following.

So, why is it called a watermark? Clearly the term itself has caused confusion. Google has a similar capability, except it’s a synthetic watermark for images (SynthID), and that seems to have come across as legitimate and sensible. Of course the research papers use “watermarking” as a term, but Anthropic could have wrapped it better in some marketing prose, alongside some imagery to confirm. At the same time the blog post itself is great at assuaging concerns developers may have, it’s just that many probably didn’t read or understand it after hearing about the term itself.

And this isn’t to blame developers! When was the last time you yourself deeply research and read a new topic, instead asking your coworkers, or reading a tweet about a new thing. People form impressions quickly, and it’s on you to challenge them.

In either case, the word they chose to go forward with strongly conditioned how people were gonna respond, and in this case, with really painful consequences surrounding the trust people have in what Anthropic is doing.

When describing something new, choose wisely

So what’s the lesson in all of this?

Developers are people, and people form mental models quickly based on the information available to them. And those models are often wrong but useful!

When marketing complex technical products to developers, please place an extreme amount of importance on what words and mental frameworks you are giving them to use. If you don’t, you may accidentally let your developers make too many assumptions about your product to great effect.

Sometimes writing works, sometimes visual explanations are most helpful, but something is better than nothing.

And, that with existing products, you may already have a bunch of incorrect assumptions made by developers lying around. So address them head on, and offer clear alternatives.

And if you product is not commensurate with any metaphor, then invest time in thinking about why and what can you do to change that, lest you want a repeat of the watermarking fiasco!