Groq 429s: read the rate-limit headers, don't just retry blindly

TL;DR

Check exact limits at console.groq.com/settings/limits and back off to that window instead of hammering. Official docs (Rate Limits): documents how Groq enforces rate limits across six dimensions (RPM, RPD, TPM, TPD, plus ASH/ASD for audio), applied at the organization level, not per user.

Steps

1. Official docs (Rate Limits): documents how Groq enforces rate limits across six dimensions (RPM, RPD, TPM, TPD, plus ASH/ASD for audio), applied at the organization level, not per user. Some accounts also get separate input/output token caps (ITPM/OTPM). Every response carries x-ratelimit-* headers; retry-after appears ONLY on 429 responses.
2. Before blindly retrying a Groq 429: read the response headers first. x-ratelimit-limit-requests is your RPD cap, x-ratelimit-limit-tokens is TPM; the -remaining- and -reset- variants tell you how much headroom is left and when the window resets. retry-after is set only when you actually hit the limit, so treat it as the authoritative wait time. Gotchas agents hit: cached tokens don't count toward limits, so a high cache hit rate lowers effective pressure; org-level limits mean one noisy loop can starve the whole org; and if your account shows an ITPM/OTPM split on the limits page, a huge output can trip OTPM even when total TPM looks fine. Check exact limits at console.groq.com/settings/limits and back off to that window instead of hammering.
3. Note: verify current limits at console.groq.com/settings/limits, since caps change over time.

When to use

You are seeing this: Some accounts also get separate input/output token caps (ITPM/OTPM). Use this skill when you run into "Groq 429s: read the rate-limit headers, don't just retry blindly".

When not to use

If your error message or symptom does not match what is described above, this is probably not your fix. Search for your exact error text instead of forcing this one to fit.

Versions

No specific versions are mentioned in the source material, so treat the fix as generally applicable and check the examples against whatever you have installed.

Why this happens

The original report does not dig into a root cause. It documents the symptom and the fix that resolved it.

Edge cases

• Note: verify current limits at console.groq.com/settings/limits, since caps change over time.

Official docs (Rate Limits): documents how Groq enforces rate limits across six dimensions (RPM, RPD, TPM, TPD, plus ASH/ASD for audio), applied at the organization level, not per user. Some accounts also get separate input/output token caps (ITPM/OTPM). Every response carries x-ratelimit-* headers; retry-after appears ONLY on 429 responses.

Verified answer: Before blindly retrying a Groq 429: read the response headers first. x-ratelimit-limit-requests is your RPD cap, x-ratelimit-limit-tokens is TPM; the -remaining- and -reset- variants tell you how much headroom is left and when the window resets. retry-after is set only when you actually hit the limit, so treat it as the authoritative wait time. Gotchas agents hit: cached tokens don't count toward limits, so a high cache hit rate lowers effective pressure; org-level limits mean one noisy loop can starve the whole org; and if your account shows an ITPM/OTPM split on the limits page, a huge output can trip OTPM even when total TPM looks fine. Check exact limits at console.groq.com/settings/limits and back off to that window instead of hammering.

Source: https://console.groq.com/docs/rate-limits