prompt cache savings not counted error cost report wrong fails

Not specified by the original author.

TL;DR: For most Claude models, only uncached input tokens count toward the per-minute input token limit. Prompt caching is a rate-limit strategy, not just a cost strategy. Cache repeated content: system instructions, tool definitions, large context documents, conversation history. Note the exception: Claude Haiku 3.5 also counts cache reads toward ITPM.

ITPM counts uncached tokens: cache to raise your ceiling

Rate limits are measured per model in RPM, ITPM, and OTPM. For most models, what counts toward ITPM is inputtokens plus cachecreationinputtokens. Tokens served from cache (cachereadinput_tokens) do not count. The documented example: with a 2M ITPM limit and an 80 percent cache hit rate, you can effectively process 10M total input tokens per minute.

What to do

1. Cache repeated content: system instructions, tool definitions, large context documents, conversation history. Put stable content early so cache breakpoints hit.
2. Note the exception: Claude Haiku 3.5 also counts cache reads toward ITPM. Do not assume the cache discount on every model; check the footnotes for yours.
3. Remember maxtokens does not factor into OTPM calculations. There is no rate-limit downside to setting a higher maxtokens.
4. ITPM is estimated at request start and adjusted during the request to reflect actual input tokens used. Short bursts can still trip the limit even under the per-minute average.
5. Monitor your cache hit rate on the Console Usage page and tune breakpoints until the hit rate climbs.

The trap

Optimizing RPM while ITPM is the binding constraint, or vice versa. Read the 429 message: it names the exceeded limit, so fix that dimension. The other trap: caching nothing and requesting a limit increase, when a cache breakpoint would have multiplied your effective limit for free.

Checklist

• When you 429 on ITPM, the first fix is caching, the second fix is spreading across models, the third is a limit increase. In that order.
• Verify the cache discount applies to your model before depending on it in capacity planning.

When to use this

• This covers exactly what the title says: Anthropic cache-aware ITPM.
• You are setting this up for the first time, or auditing an existing setup.
• You want the key gotchas in one place before you start.

When not to use this

• You are doing a different workflow with Anthropic; these steps are specific to the title above.
• You need the full reference docs; this is the short path, not the manual.

Compatibility

• Mentions 3.5 in the original; behavior may differ on much older or newer releases.

Anthropic's rate-limit reference documents cache-aware ITPM (only uncached input tokens count for most models; Haiku 3.5 counts cache reads), that max_tokens does not affect OTPM, and the 10M-effective-throughput example. Verified live at https://platform.claude.com/docs/en/api/rate-limits.

Not specified by the original author.

TL;DR: cached_tokens sits at zero and the expected discount never appears. Caching is automatic but picky: the prefix must be long enough and byte-identical. Confirm the cause Check the documented requirements against your requests: 1. Shorter prompts never cache.

The symptom

usage.prompt_tokens_details.cached_tokens is 0 on calls that look identical, and the bill shows no cached-input discount. Caching is automatic on supported models, so zero hits means the requirements are not met.

Confirm the cause

Check the documented requirements against your requests:

1. Minimum cacheable length. 1,024 tokens for GPT-5.6 and later; varies by model and request settings for earlier models. Shorter prompts never cache. Hidden system tokens do not count toward the minimum.
2. Exact prefix match. The cache matches the longest previously computed prefix. Any early difference (timestamp, request ID, reordered tools, regenerated instructions) breaks the match from that point. Static content first and byte-identical; volatile data last.
3. What counts. Messages, images, tool definitions, and structured output schemas can all be cached and count toward the minimum. Changing the tool list between calls resets the match.
4. Reporting granularity. Reported cached_tokens is computed from the last matched breakpoint and rounded down to the nearest multiple of 128. Near-threshold prompts can show small or zero values even when mostly matching.

The fix

• Stable prefix first: instructions, examples, tool definitions, images byte-stable; user data, timestamps, and retrieval results last.
• Get the shared prefix over the minimum. Six hundred identical tokens never cache no matter how stable they are.
• Keep model, tools, and image detail identical across the calls you want cached together.

Verify the fix

Log cachedtokens per request and compute the hit ratio over a day. Identical-prefix calls should show large cachedtokens values after restructuring. For persistent misses, diff prefixes byte by byte; the first difference is the culprit.

When to use this

• This covers exactly what the title says: OpenAI prompt caching not saving money.
• You are setting this up for the first time, or auditing an existing setup.
• You want the key gotchas in one place before you start.

When not to use this

• You are doing a different workflow with OpenAI; these steps are specific to the title above.
• You need the full reference docs; this is the short path, not the manual.

Compatibility

• Mentions 5.6 in the original; behavior may differ on much older or newer releases.

OpenAI's prompt caching guide states the minimum cacheable prompt length is 1,024 tokens for GPT-5.6 and later (varying earlier), hidden system tokens do not count toward it, caching matches the longest previously computed prefix, and reported cached_tokens rounds down to the nearest multiple of 128. Messages, images, tools, and schemas can be cached and count toward the minimum (https://platform.openai.com/docs/guides/prompt-caching).

Not specified by the original author.

TL;DR

Check the minimum: the cacheable prefix must reach at least 1,024 tokens for current models. Caching works on prefixes, so anything that changes per request must sit at the end. Keep the prefix byte-identical across requests. If cached tokens stay at zero, the prefix is not stable or is under the minimum; fix the structure, not the API call.

When to use

You are seeing this: Measure the baseline first: log usage.prompt\tokens\details on a sample of requests and note what is billable today. Use this skill when you run into "Roll out OpenAI prompt caching: stable prefixes, verify cached_tokens, measure savings".

When not to use

If your error message or symptom does not match what is described above, this is probably not your fix. Search for your exact error text instead of forcing this one to fit.

Versions

No specific versions are mentioned in the source material, so treat the fix as generally applicable and check the examples against whatever you have installed.

Prompt caching rollout

The steps

1. Measure the baseline first: log usage.prompt_tokens_details on a sample of requests and note what is billable today. You cannot prove savings without a before number.
2. Restructure prompts so the stable content comes first: system instructions, tool definitions, few-shot examples, then the variable user content last. Caching works on prefixes, so anything that changes per request must sit at the end.
3. Check the minimum: the cacheable prefix must reach at least 1,024 tokens for current models. Short prompts do not qualify; do not expect caching on a 200-token system message.
4. Keep the prefix byte-identical across requests. One dynamic timestamp or user id injected early in the prompt breaks the cache for everything after it. Move volatile fields to the end or out of the prompt entirely.
5. Verify with cached_tokens in the usage object. If cached tokens stay at zero, the prefix is not stable or is under the minimum; fix the structure, not the API call.
6. For workloads that need separate cache accounting or routing control, use a stable prompt_cache_key. This keeps distinct workflows from evicting each other's cached prefixes.

The trap

Assuming caching is automatic. The API caches opportunistically on matching prefixes; a prompt that shuffles content order per request gets zero hits and the team concludes caching "does not work."

Checklist

• Baseline token usage logged before the rollout.
• Stable prefix of 1,024+ tokens, volatile content last.
• cached_tokens verified nonzero in production traffic.
• promptcachekey set where workflows share a model.
• Savings re-measured monthly; prompt edits re-verified for cache hits.

OpenAI's prompt caching guide documents the 1,024-token minimum, discounts up to 90%, cachedtokens in usage, and promptcache_key. Verified live at https://developers.openai.com/api/docs/guides/prompt-caching.