DeepInfra rate limits are per model concurrency, not requests per minute

TL;DR: If DeepInfra returns 429 with Rate limited, check concurrency first, not RPM: the default cap is 200 simultaneous requests per model. Long requests hit the cap at modest rates, so throttle with a token bucket and retry after a short delay. A 429 while under the limit usually means the model is busy and scaling up; a brief wait resolves it.

Fix

1. If DeepInfra returns 429 with Rate limited, check concurrency first, not RPM: the default cap is 200 simultaneous requests per model. Expected: this specific failure stops.

2. Context: DeepInfra docs Rate Limits (deepinfra/docs account/rate-limits.mdx): The default limit is 200 concurrent requests per model, measured per model, so two models in parallel allow 400 total. Expected: this specific failure stops.

Details

Long requests hit the cap at modest rates, so throttle with a token bucket and retry after a short delay. If your steady state needs more headroom, request a limit increase rather than spreading calls across models. It is a concurrency limit, not per-minute: at 10s average request duration the 200-concurrent cap equals roughly 1,200 RPM. Exceeding it returns HTTP 429 with a Rate limited message. The docs note you can also get occasional 429s on busy models while under the limit, when auto-scaling has not kicked in yet.

When to use

You hit exactly this: DeepInfra rate limits are per model concurrency, not requests per minute.

When not to use

A different error, or the same symptom in a different tool. This page only covers the failure above.

Compatibility

the tool named in the title.

Root cause

A 429 while under the limit usually means the model is busy and scaling up; a brief wait resolves it.

DeepInfra docs Rate Limits (deepinfra/docs account/rate-limits.mdx): The default limit is 200 concurrent requests per model, measured per model, so two models in parallel allow 400 total. It is a concurrency limit, not per-minute: at 10s average request duration the 200-concurrent cap equals roughly 1,200 RPM. Exceeding it returns HTTP 429 with a Rate limited message. The docs note you can also get occasional 429s on busy models while under the limit, when auto-scaling has not kicked in yet. Resolved: If DeepInfra returns 429 with Rate limited, check concurrency first, not RPM: the default cap is 200 simultaneous requests per model. Long requests hit the cap at modest rates, so throttle with a token bucket and retry after a short delay. A 429 while under the limit usually means the model is busy and scaling up; a brief wait resolves it. If your steady state needs more headroom, request a limit increase rather than spreading calls across models. Source: https://github.com/deepinfra/docs/blob/HEAD/account/rate-limits.mdx