You wrote ten thousand tokens. You paid for a hundred and five thousand.

Two engineers on the same team, the same editor, the same models, the same $320 a month in subscriptions. One hits the usage limit on the twelfth. The other has never hit it. Neither writes more code than the other.

The difference is not the vendor, the model, or the tier. It is whether the expensive part of their prompt holds still.

The bill everyone quotes

Every video about AI costs opens with some version of this. Twenty dollars for the editor, a hundred for one chat subscription, two hundred for the other. The arithmetic is right — Cursor Pro is $20/mo, Claude Max 5x is $100, ChatGPT Pro is $200 — and the conclusion drawn from it is almost always wrong.

The conclusion is: replace the stack. Run the model locally, pull the weights, buy the card instead of renting it. All of that answers one question — how do I pay less per token? — and the price of a token is the single number in this system that is public, stable, and falling.

The number that actually sets your bill is not on any pricing page.

A model has no memory

A language model does not remember your conversation. There is no session sitting on a server holding your last question. Nothing carries over between calls.

So turn two re-sends turn one. Turn three re-sends one and two. By turn twenty you are shipping all nineteen previous turns up the wire again, and paying to have them read again, every single time.

Take a conversation where each turn adds about 500 tokens. Twenty turns in:

That is not an estimate and not a benchmark. It is arithmetic. With w tokens added per turn across n turns you write w·n, and the model reads:

w · n(n+1)/2

so the multiplier is just (n+1)/2. Twenty turns gives 10.5. Forty gives 20.5. A hundred gives 50.5.

The cost of a conversation grows with the square of its length, while the conversation itself feels like it is growing one turn at a time. That gap — between how it feels and how it bills — is the whole problem.

Where the multiplier lands

It does not spread evenly. Almost everything in a request is byte-for-byte identical from one call to the next: the system prompt, the tool definitions, the document you pasted in at turn one. The only genuinely new tokens in turn twenty are the ones you typed thirty seconds ago.

Both major providers will sell you that repeat at a discount, and the size of it is the part people skip past.

From Anthropic's prompt caching documentation. On Claude Fable 5.1 and Mythos 5.1 the read multiplier drops further, to 0.025×. OpenAI's own worked example uses the same two numbers — 0.1 to read, 1.25 to write — and describes the effect as up to 90% off.

One detail worth stating precisely, because it is commonly misread: the cache write is not an additional fee stacked on top. Each input token is billed at exactly one of the three rates — uncached, cached, or cache-write.

So the expensive, repetitive 90% of your input has a price ten times lower than the one most people are paying for it.

The catch, which is the whole thing

Caching is prefix-based. The provider matches the longest run of tokens, starting from the very beginning of your prompt, that is identical to something it has already seen. The moment it finds a difference, everything after that point is a miss.

Not the changed part. Everything after the changed part.

system: You are a helpful assistant.
        The current time is 14:07:33.     ← 13 characters
tools:  [ ... 900 tokens ... ]
docs:   [ ... 6000 tokens ... ]
user:   What changed in this file?

That timestamp seemed useful. Underneath it sit 900 tokens of tool definitions and 6,000 tokens of documents, every one identical to the previous call. All of them miss. All of them bill at full rate, because one line above them moved.

Why it works that way, and why it will not be relaxed

What gets cached is not your text. It is the internal state the model built while reading your text, and every position in that state depends on every position before it. Position 6,900 was computed from positions 0 through 6,899. Change one token and every piece of state after it was derived from something that no longer matches, so it has to be computed again.

There is no way to patch the middle. The boundary only ever walks forward.

This is a property of how the model reads, not a limitation of anyone's caching implementation — which is why no vendor is going to ship a fix for it, and why prompt order is worth treating as an architectural decision rather than a formatting one.

Same models. Same vendors. Same tokens. Ten times the input cost.

A floor that catches people out

OpenAI only caches a prefix of at least 1,024 tokens. Their guidance notes that padding a 221-token prefix up to that threshold pays for itself within about ten requests.

Which is genuinely counterintuitive: a short prefix is not the cheap option. It is the option with no discount available at all.

Back to the two engineers

One of them has a system prompt that has not changed in a month, tools declared once, documents appended in a fixed order, and the question at the bottom. Roughly 90% of their input is a cache read at a tenth of the price.

The other injects the current time into the system prompt, because that seemed useful, and reorders retrieved documents by relevance on every call, because that also seemed useful. Both are reasonable engineering decisions. Both sit above the expensive part of the prompt.

Where else this applies

When Cursor moved its Pro plan from "500 requests a month" to "$20 of usage a month", that was not really a price rise. It was the costume coming off. The subscription was always a token budget; charging in requests just hid the meter. A request whose prefix misses the cache spends that budget ten times faster than one that hits.

And running the model yourself does not escape any of this. We worked through those economics in episode two: local inference swaps a per-token price for a per-second one, and the meter runs whether or not you are using it. A stateless model is still stateless on your own hardware — it re-reads the entire conversation on every turn exactly as before. The only thing that changed is whether you pay in dollars or in seconds of a card you have already bought.

What to look at next

The useful question was never which vendor is cheapest. It is what is actually in your prefix, and whether any of it moves.

Most people have never counted. Print the exact bytes you send on two consecutive calls and diff them:

diff <(cat call-1.json) <(cat call-2.json)

The first line that differs is where your discount stops. Everything below it is being bought at full price, on every single call.

Sources

• Anthropic, prompt caching pricing — cache write 1.25× (5 min) and 2× (1 hour), cache read 0.1×, dropping to 0.025× on Claude Fable 5.1 and Mythos 5.1. https://platform.claude.com/docs/en/build-with-claude/prompt-caching
• OpenAI, prompt caching guide — the same 0.1 cached-input and 1.25 cache-write multipliers in their own worked example, described as up to 90% off; cache write is not additive; 1,024-token minimum prefix. https://developers.openai.com/api/docs/guides/prompt-caching
• Cursor pricing — Pro at $20/mo, and the shift from request counts to usage. https://cursor.com/pricing
• OpenAI, ChatGPT Pro tiers — Pro at $200/mo. https://help.openai.com/en/articles/9793128-about-chatgpt-pro-tiers

The re-read multiplier is not cited, because it is not a claim about the world. With w tokens added per turn over n turns you write w·n and the model reads w·n(n+1)/2, so the multiplier is (n+1)/2. At n=20 that is 10.5. You can check it on a calculator in about thirty seconds.