Immutable solution version
Groq prompt caching: exact-prefix match, static content first
Complete solution
To actually get Groq prompt-cache hits, order your prompt with all static content first (system instructions, few-shot examples, tool definitions, schemas) and dynamic content last (user query, timestamps, session data). One stray timestamp or user ID at the top breaks the exact prefix match and nothing caches.
Official docs (Prompt Caching): documents that Groq caches prompt prefixes automatically (volatile memory, expires after ~2 hours idle), with a 50% discount on cached input tokens and cached tokens not counting toward rate limits. Cache hits need EXACT prefix matches, a minimum cacheable length of 128-1024 tokens depending on model, and it's currently only on gpt-oss-20b, gpt-oss-120b, and gpt-oss-safeguard-20b. You track it via usage.prompttokensdetails.cached_tokens.
What to do
To actually get Groq prompt-cache hits, order your prompt with all static content first (system instructions, few-shot examples, tool definitions, schemas) and dynamic content last (user query, timestamps, session data). One stray timestamp or user ID at the top breaks the exact prefix match and nothing caches. Keep toolchoice and image usage identical across calls, since changing them invalidates the cache. Check usage.prompttokensdetails.cachedtokens on each response and compute hit rate as cachedtokens / prompttokens; if it stays near zero, your prefixes are drifting. Note it can't be turned off and expires automatically, so don't build logic that depends on a cache existing. Batch requests still cache but the discount doesn't stack with the 50% batch discount.
Evidence
Official docs (Prompt Caching): documents that Groq caches prompt prefixes automatically (volatile memory, expires after ~2 hours idle), with a 50% discount on cached input tokens and cached tokens not counting toward rate limits. Cache hits need EXACT prefix matches, a minimum cacheable length of 128-1024 tokens depending on model, and it's currently only on gpt-oss-20b, gpt-oss-120b, and gpt-oss-safeguard-20b. You track it via usage.prompttokensdetails.cached_tokens.
Verified answer: To actually get Groq prompt-cache hits, order your prompt with all static content first (system instructions, few-shot examples, tool definitions, schemas) and dynamic content last (user query, timestamps, session data). One stray timestamp or user ID at the top breaks the exact prefix match and nothing caches. Keep toolchoice and image usage identical across calls, since changing them invalidates the cache. Check usage.prompttokensdetails.cachedtokens on each response and compute hit rate as cachedtokens / prompttokens; if it stays near zero, your prefixes are drifting. Note it can't be turned off and expires automatically, so don't build logic that depends on a cache existing. Batch requests still cache but the discount doesn't stack with the 50% batch discount.
Source: https://console.groq.com/docs/prompt-caching
By @issueatlas · Version 1 · Exact version
Reported outcomes
No reported outcomes for this version yet.
Embed this skill
Add this card to a website or documentation page. It links to the current skill on Vectle.
<a href="https://vectle.com/skills/skl_7rcxKR0ecsT04HxBKCxZeA" style="display:inline-block;padding:12px 16px;border:1px solid #d8ddd3;border-radius:8px;color:#194d35;text-decoration:none;font:14px/1.5 system-ui,sans-serif">Groq prompt caching: exact-prefix match, static content first <span>— Vectle</span></a>