OpenAI prompt caching not saving money: the 1024-token minimum and prefix rules

OpenAI prompt caching not saving money: the 1024-token minimum and prefix rules

The symptom

usage.prompt_tokens_details.cached_tokens is 0 on calls that look identical, and the bill shows no cached-input discount. Caching is automatic on supported models, so zero hits means the requirements are not met.

Confirm the cause

Check the documented requirements against your requests:

1. Minimum cacheable length. 1,024 tokens for GPT-5.6 and later; varies by model and request settings for earlier models. Shorter prompts never cache. Hidden system tokens do not count toward the minimum.
2. Exact prefix match. The cache matches the longest previously computed prefix. Any early difference (timestamp, request ID, reordered tools, regenerated instructions) breaks the match from that point. Static content first and byte-identical; volatile data last.
3. What counts. Messages, images, tool definitions, and structured output schemas can all be cached and count toward the minimum. Changing the tool list between calls resets the match.
4. Reporting granularity. Reported cached_tokens is computed from the last matched breakpoint and rounded down to the nearest multiple of 128. Near-threshold prompts can show small or zero values even when mostly matching.

The fix

• Stable prefix first: instructions, examples, tool definitions, images byte-stable; user data, timestamps, and retrieval results last.
• Get the shared prefix over the minimum. Six hundred identical tokens never cache no matter how stable they are.
• Keep model, tools, and image detail identical across the calls you want cached together.

Verify the fix

Log cachedtokens per request and compute the hit ratio over a day. Identical-prefix calls should show large cachedtokens values after restructuring. For persistent misses, diff prefixes byte by byte; the first difference is the culprit.

OpenAI's prompt caching guide states the minimum cacheable prompt length is 1,024 tokens for GPT-5.6 and later (varying earlier), hidden system tokens do not count toward it, caching matches the longest previously computed prefix, and reported cached_tokens rounds down to the nearest multiple of 128. Messages, images, tools, and schemas can be cached and count toward the minimum (https://platform.openai.com/docs/guides/prompt-caching).