Prompt Caching in LLMs, Measured on Our Own Bill

The token report for one pipeline run landed in front of me and the first number I saw was 7,300,000. One topic. Seven pieces of content. Seven point three million tokens. That is the kind of number you see right before someone suggests shutting the project down.

The bill for that run, at API list prices: $8.12.

The distance between the panic and the price is prompt caching. I have spent months running a content pipeline on Claude, and the numbers from inside that pipeline taught me more about caching than any pricing page — including one line item nobody warned me about, which turned out to be the biggest line on the bill.

The setup, briefly

Our pipeline is a TypeScript orchestrator that spawns around 18 Claude agent runs per topic: research agents that build a fact sheet, writers that produce articles against it, a verifier that checks every claim, plus hub and glossary generation. One full run produces 5 articles, a glossary entry and a hub page. Everything below comes from one benchmark run of that pipeline on Claude Sonnet 4.6, cross-checked against two more runs from the same day.

A token total is an impression, not information

Here is the same 7.3 million, decomposed — four lines from the run report:

The scary headline number is 85 % cache reads — tokens billed at one tenth of the input price. Only 0.2 % of the run was full-price input. If you take one habit away from this article, take this one: never judge an agentic run by its token total. The total is an impression; the four-line breakdown is the information. We learned this the embarrassing way — our own first-generation run reports tracked cache only as a run total, not per phase, precisely because we hadn't yet understood it was the number that mattered.

How does prompt caching reduce LLM API costs?

The headline benefit of prompt caching, measured on our own bill instead of a pricing page: it cut this run's cost to a third — $8.12 instead of roughly $24. Priced at Sonnet 4.6 list rates, the run looks like this:

Now the counterfactual. Without caching, those 7.16 million non-output tokens would all be plain input: 7.16M × $3 = $21.48, plus the same $2.67 of output — about $24. Caching cut this run's cost to a third. Per article, that is $1.43 instead of roughly $4.25.

Why is nearly everything a re-send in the first place? Because that is what an agentic loop is. Every time an agent calls a tool and continues, the whole conversation goes back to the model — system prompt, instructions, fact sheet, everything, again. That is how prompt caching works: the provider stores your prompt prefix, and when a later call starts with the same prefix, those tokens are read from cache at 10 % of the input price instead of being processed at full price. An 18-agent pipeline is a machine for re-sending the same context hundreds of times, which is exactly why caching moves the bill by 3× and not by some rounding amount.

86.6 % hit rate is architecture, not luck

Our hit rate for that run was 86.6 %. Careful, this is not the 85 % from the breakdown table: that one was cache reads as a share of all tokens including output; hit rate measures cache reads as a share of everything the model read (input + cache read + cache write) — the question "of all the context we sent, how much came from cache." Two more entities ran through the pipeline the same day, on different topics, and landed at the same shape: 6.38M/1.10M and 6.31M/0.92M cache read/write. Three runs within a few percent of each other is not luck. It is also not tuning — and that is the honest part: we never sat down to "optimize for caching."

What produced it is architectural, and it was there before we understood its billing consequences. Caching matches on a stable prefix, and our pipeline happens to be built out of stable prefixes: every agent reads its role definition from the same file, every writer gets its brief in the same format, templates are read from one place at runtime instead of being pasted into prompts. The same properties we wanted for maintainability — single source of truth, templates referenced instead of duplicated — turned out to be exactly cache-shaped.

The reverse is equally true, and it is the failure mode to check in your own system: anything that churns early in your prompt — a timestamp, a random run ID, a reshuffled file list — breaks the prefix match from that point on, and your hit rate quietly collapses while your architecture diagram still looks perfectly cacheable.

The line item nobody warned me about

Look at the cost table again, because this is where the drawbacks of prompt caching live. The single biggest line is not output. It is cache write — $3.56, 44 % of the whole bill, more than the model charged for actually generating seven pieces of content.

Cache writes cost 25 % more than plain input ($3.75 vs $3 per MTok for the default 5-minute tier). Every cached token is an investment: you pay a premium upfront, and it pays back only if that prefix gets read again. Our ratio was about 6.5 reads per written token, so the investment returned roughly six times over. But the arithmetic has a break-even, and Anthropic's docs state it plainly: the 5-minute cache pays for itself after a single cache read (the write premium is 1.25×), the 1-hour cache after two (2×). The loss case is the cache that never gets read back — one-shot scripts and prompts that churn on every call pay the 25 % premium for nothing. Our 6.5 reads per written token clear either bar comfortably.

There is also a clock on it: at Anthropic the cache lives five minutes by default, refreshed at no extra cost every time it is used, with a paid one-hour option. A pipeline that runs its phases back to back keeps the prefix warm the whole way through. A job that fires a call every half hour re-pays the write premium every single time and reads nothing back.

Which is also the honest counterargument to this whole article: our 3× is a property of our workload, not of caching itself. An agentic pipeline re-reading stable templates hundreds of times within minutes is close to caching's best case. A chat assistant with one user and coffee-length pauses between messages sits near its worst — prefixes expire before they are re-read, and the premium buys nothing. Do not budget a 3× saving because we measured one; measure your own read/write ratio first.

Three rules we kept

Read the breakdown, never the total. Input, output, cache read, cache write — four lines or you know nothing. This 7.3M-token run cost less than a single million tokens of plain output would ($8.12 vs $15).

Hit rate is an architecture property. Stable role files, stable briefs, runtime reads from one canonical place — the maintainability rules you already believe in are the same rules that keep your prefix stable. Audit for churn near the top of your prompts.

Cache write is an investment with a break-even. One re-read pays off a 5-minute write, two pay off a 1-hour write — provided the re-read lands inside the TTL. Compute your read/write ratio once; if it sits near zero, you are donating a 25 % premium to your provider.

One honesty note to close. This run executed on a Claude subscription; the $8.12 is the API-equivalent at list prices — the number you would pay building the same thing against the API. And we did not get here by designing for caching. We got here by designing for maintainability and discovering, in the billing breakdown, that the two are mostly the same thing.