Groq prompt caching: exact-prefix match, static content first

TL;DR

Check usage.prompttokensdetails.cachedtokens on each response and compute hit rate as cachedtokens / prompt_tokens; if it stays near zero, your prefixes are drifting.

Steps

1. Official docs (Prompt Caching): documents that Groq caches prompt prefixes automatically (volatile memory, expires after ~2 hours idle), with a 50% discount on cached input tokens and cached tokens not counting toward rate limits. Cache hits need EXACT prefix matches, a minimum cacheable length of 128-1024 tokens depending on model, and it's currently only on gpt-oss-20b, gpt-oss-120b, and gpt-oss-safeguard-20b. You track it via usage.prompttokensdetails.cached_tokens.
2. To actually get Groq prompt-cache hits, order your prompt with all static content first (system instructions, few-shot examples, tool definitions, schemas) and dynamic content last (user query, timestamps, session data). One stray timestamp or user ID at the top breaks the exact prefix match and nothing caches. Keep toolchoice and image usage identical across calls, since changing them invalidates the cache. Check usage.prompttokensdetails.cachedtokens on each response and compute hit rate as cachedtokens / prompttokens; if it stays near zero, your prefixes are drifting. Note it can't be turned off and expires automatically, so don't build logic that depends on a cache existing. Batch requests still cache but the discount doesn't stack with the 50% batch discount.

When to use

You are seeing this: You track it via usage.prompttokensdetails.cached_tokens. Use this skill when you run into "Groq prompt caching: exact-prefix match, static content first".

When not to use

If your error message or symptom does not match what is described above, this is probably not your fix. Search for your exact error text instead of forcing this one to fit.

Versions

No specific versions are mentioned in the source material, so treat the fix as generally applicable and check the examples against whatever you have installed.

Why this happens

The original report does not dig into a root cause. It documents the symptom and the fix that resolved it.

Edge cases

• Note it can't be turned off and expires automatically, so don't build logic that depends on a cache existing.

Official docs (Prompt Caching): documents that Groq caches prompt prefixes automatically (volatile memory, expires after ~2 hours idle), with a 50% discount on cached input tokens and cached tokens not counting toward rate limits. Cache hits need EXACT prefix matches, a minimum cacheable length of 128-1024 tokens depending on model, and it's currently only on gpt-oss-20b, gpt-oss-120b, and gpt-oss-safeguard-20b. You track it via usage.prompttokensdetails.cached_tokens.

Verified answer: To actually get Groq prompt-cache hits, order your prompt with all static content first (system instructions, few-shot examples, tool definitions, schemas) and dynamic content last (user query, timestamps, session data). One stray timestamp or user ID at the top breaks the exact prefix match and nothing caches. Keep toolchoice and image usage identical across calls, since changing them invalidates the cache. Check usage.prompttokensdetails.cachedtokens on each response and compute hit rate as cachedtokens / prompttokens; if it stays near zero, your prefixes are drifting. Note it can't be turned off and expires automatically, so don't build logic that depends on a cache existing. Batch requests still cache but the discount doesn't stack with the 50% batch discount.

Source: https://console.groq.com/docs/prompt-caching