how to implement rate limiting that works

how to implement rate limiting that works

TL;DR

Rate limiting only works if the counter is shared, the key identifies the abuser, and the limit is enforced before expensive work happens. Use a token bucket in a shared store keyed on the authenticated identity (falling back to IP), return 429 with a retry hint, and test it under real concurrency. A limiter that lives in one process's memory is decoration.

how to implement rate limiting that works

Use this when

• You are adding throttling to a public or partner API
• Login, signup, or password-reset endpoints lack abuse protection
• A scraper or aggressive client is hammering your service
• An agent is reviewing whether existing limits actually trigger
• You need per-customer quotas on top of abuse protection

Not for this skill when

• You are buying DDoS protection (infrastructure territory)
• You are designing CAPTCHAs or bot challenges
• The question is about billing meters or plan quotas as a product feature
• You need request prioritization under load rather than rejection

Steps

1. Pick the key that identifies the abuser

Key on the authenticated user or API key name first; fall back to IP only for anonymous endpoints. IP-only limiting punishes everyone behind one NAT and misses attackers rotating addresses.

Expected: a written decision per endpoint for what the key is, and no authenticated endpoint keyed solely on IP.

2. Put the counter in a shared store

Every app instance must see the same count, so keep it in redis or equivalent with atomic increments and expiries. In-process maps split the budget across instances and reset on deploy.

redis-cli --raw incr "ratelimit:test-key" | head -1

Expected: the counter increments atomically and you can set a TTL on the key. If two app servers disagree about the count, the store is not shared.

3. Use a token bucket, not just a fixed window

A fixed window lets a client fire the whole quota in the first second of each window. A token bucket refills steadily and smooths bursts while still allowing short ones.

Expected: the limiter config names an algorithm (token bucket or sliding window) with a refill rate and burst size, and the burst size is a deliberate choice, not an accident.

4. Enforce before the expensive work

The limit check must run before auth lookups, DB queries, or model inference. A limiter placed after the costly call still lets attackers burn your resources.

Expected: in the request pipeline, the throttle middleware sits ahead of the handler logic. Review the middleware order, not just the config values.

5. Return 429 with a retry hint

Rejected requests get status 429 and a Retry-After header so well-behaved clients back off instead of hammering harder. Log the rejections for abuse review.

for i in $(seq 1 120); do curl -s -o /dev/null -w "%{http_code}
" https://example.com/api/v1/status; done | sort | uniq -c

Expected: the tail of the run shows 429s, and the 429 responses carry Retry-After. If you never see a 429, the limit is not wired into this path.

6. Load-test the limiter itself

Hit the endpoint from several concurrent clients and confirm the aggregate stays near the configured rate and that legitimate traffic patterns still pass.

Expected: a test script in the repo that reproduces the limit, so the next refactor cannot silently drop it.

Variant: login and auth endpoints

These deserve the strictest limits and the harshest keys: per-account throttling plus per-IP, with progressive delays. Credential stuffing is the threat, so a global generous limit is the wrong shape here.

Variant: webhook receivers

Your own webhooks need limits too, keyed per sender, or a misbehaving partner's retry storm becomes your outage. Verify the signature before counting, so attackers cannot burn a partner's quota.

Variant: GraphQL and expensive queries

One GraphQL request can cost as much as hundreds of REST calls, so count query complexity, not just requests. Assign costs per field and limit the budget per time window.

Why this happens

Most rate limit tutorials show a fixed window in process memory because it fits in a blog post, and teams copy it into production where multiple instances and real attackers live. The limiter then fails exactly when it matters: under distributed load, against a client that noticed the seams. Getting the key, the store, and the placement right is the whole job; the algorithm is the easy part.

Edge cases and pitfalls

• Clock skew between instances breaks window math; prefer server-side atomic ops over client-computed windows.
• Legitimate bulk clients (mobile apps on launch day, partner backfills) will trip naive limits; offer an allowlist or higher tier, not a disabled limiter.
• Retry-After values that are too generous teach scrapers your exact budget; keep them coarse.
• Rate limiting login by IP alone lets an attacker lock out real users; pair IP limits with per-account limits and alerting.
• Do not rate limit your health checks or you will confuse your own monitoring.

Provenance

Resolved from the public thread: https://vectle.com/posts/pst0mNazyiqBJheJ3Ttxjdyg