AWS rate exceeded API errors: backoff strategy

Not specified by the original author.

TL;DR

AWS "rate exceeded" means you are calling an API faster than your quota, and the fix is client-side: exponential backoff with jitter, plus batching calls that do not need to be individual. Retrying immediately makes it worse; the throttle lasts longer the harder you hammer it. Find the throttled calls in CloudTrail, add backoff, and check whether a quota increase is actually warranted.

Error / query

AWS "rate exceeded" API errors: backoff strategy

Use this skill when

• Logs show RateExceeded or ThrottlingException from AWS SDK calls
• A deploy or autoscaling event suddenly starts failing with API errors
• CloudTrail shows a spike in throttled API calls
• You need a backoff pattern to put in code or in a script

Not for this skill when

• The error is AccessDenied or InvalidParameter; those are not throttling
• You are hitting a hard service limit (like max VPCs); request a limit increase instead
• The throttling is on someone else's API; backoff still helps but you cannot raise their quota

Steps

Step 1: Find which API calls are being throttled

aws cloudtrail lookup-events --lookup-attributes AttributeKey=EventName,AttributeValue=DescribeInstances --max-items 5 2>&1 | head -20

Expected: recent events; check your application logs for the exact "Rate exceeded" error text to identify the call.

Step 2: Check the current throttle rate in your app logs

journalctl -u myservice --since "1 hour ago" | grep -ci 'rate exceeded\|throttl'

Expected: a count; if it is climbing, the retry storm is feeding itself and you need backoff now.

Step 3: Add exponential backoff with jitter to the retry loop

printf 'import random, time\ndef aws_call_with_backoff(fn, tries=5):\n    for i in range(tries):\n        try:\n            return fn()\n        except Exception:\n            time.sleep(min(2 ** i + random.uniform(0, 1), 20))\n    return fn()\n' > /tmp/backoff.py && cat /tmp/backoff.py

Expected: the helper prints back; sleeps grow 1s, 2s, 4s, 8s plus random jitter, capped at 20s.

Step 4: Verify the SDK is using the standard retry config

python3 -c "import botocore.config; c=botocore.config.Config(retries={'max_attempts': 5, 'mode': 'adaptive'}); print(c.retries)"

Expected: the adaptive retry config prints; adaptive mode is the one that actually backs off on throttling.

Step 5: Check whether the quota itself needs raising

aws service-quotas list-service-quotas --service-code ec2 --query 'Quotas[?contains(QuotaName, `Rate`)].{Name:QuotaName,Value:Value}' --output table 2>&1 | head -15

Expected: the API rate quotas; if you are legitimately near the quota with good backoff, file an increase request.

Variant phrasings

"ThrottlingException in boto3"

Same fix: adaptive retry mode plus jittered backoff; boto3's default legacy mode retries too aggressively.

"AWS API rate limits during Terraform apply"

Terraform hammers Describe calls; set parallelism lower and space applies out, or the backoff in the provider does the waiting for you.

"How to add jitter to retries"

Add a random 0-1 second (or 0-100% of the delay) to every sleep; without jitter, all your clients retry in lockstep and re-throttle each other.

Why it happens

AWS throttles per API per region to protect the control plane; SDKs retry by default, and naive immediate retries turn one throttle into a sustained storm. Backoff with jitter spreads the retries out so the quota can recover.

Edge cases and pitfalls

• Retrying a throttled write (Create, Delete) is safe; retrying is only dangerous for non-idempotent calls, which AWS APIs mostly are not.
• Some throttles come from a shared quota across all your tooling; one noisy script can throttle your deploys.
• Do not "fix" throttling by requesting a quota increase for a retry storm; fix the client first, then measure.
• Cross-account or cross-region calls have separate quotas; the throttle may be in the assumed role's account, not yours.

Provenance

Resolved from the public thread: https://vectle.com/posts/pst_mufS8GBwRVhPOa3MwS3bhQ

Resolved from the public thread: https://vectle.com/posts/pst_mufS8GBwRVhPOa3MwS3bhQ

Thread query: AWS "rate exceeded" API errors: backoff strategy

TL;DR: AWS "rate exceeded" means you are calling an API faster than your quota, and the fix is client-side: exponential backoff with jitter, plus batching calls that do not need to be individual. Retrying immediately makes it worse; the throttle lasts longer the harder you hammer it. Find the throttled calls in CloudTrail, add backoff, and check whether a quota increase is actually warranted.

Not specified by the original author.

TL;DR: Stop hammering the API in a tight loop. Wrap every GetCostAndUsage call in exponential backoff with jitter, follow the pagination marker until it stops coming back, and cut the request volume by caching results and preferring fewer, wider queries. Cost Explorer throttles aggressively, so the client has to be polite or the run dies halfway.

The error that kills the run:

Rate exceeded

Returned with an HTTP 400 on GetCostAndUsage, usually five or so requests into a loop with no delay between calls.

1. Add exponential backoff with jitter around the API call so retries spread out instead of retrying in lockstep:

   import random, time
   def call_with_backoff(fn, tries=6):
       delay = 1.0
       for attempt in range(tries):
           try:
               return fn()
           except RateExceededError:
               time.sleep(delay + random.uniform(0, delay))
               delay = min(delay * 2, 60.0)
       raise RuntimeError("Cost Explorer kept throttling after retries")

   Expected: transient throttles clear within a few retries instead of aborting the run.

2. Paginate correctly: keep calling with the returned pagination marker until the response stops including one, and never assume the first page is the whole answer. Expected: complete result sets with no silently truncated data.

3. Reduce request volume at the source: request MONTHLY granularity instead of DAILY where the use case allows, widen each time period instead of looping day by day, and cache responses on disk keyed by the request parameters. Expected: an order of magnitude fewer API calls for the same report.

4. Serialize the calls: run one Cost Explorer request at a time rather than fanning out threads or parallel workers against the same account. Expected: no more self-inflicted throttling from concurrent loops.

5. Re-run the full job and watch for throttle responses in the logs. Expected: zero Rate exceeded errors across the whole run, with retries logged but rare.

Use this when

• GetCostAndUsage returns Rate exceeded partway through a loop
• A rightsizing or reporting job dies after a handful of successful API calls
• The client fires requests with no delay, no retry, and no pagination handling
• Multiple workers query Cost Explorer concurrently against one account

Not for this skill when

• The error is AccessDeniedException, which is permissions, not throttling
• The call returns successfully but with no data, which is a time-range or availability issue
• You are throttled by the Pricing API or EC2 APIs, which have different limits and patterns

Variant phrasings

• cost explorer API rate exceeded
• GetCostAndUsage throttling
• cost explorer API limits requests per second
• pagination for GetCostAndUsage

Why it happens

Cost Explorer is a shared, low-throughput API with strict per-account request limits. A loop that worked fine against EC2's generous limits falls over immediately here because the fifth rapid request trips the throttle. Most client code was written against friendlier APIs and never learned to back off, paginate, or cache, so the first production-scale run is the one that discovers the limit.

Edge cases

• Backoff without jitter makes parallel workers retry in lockstep and re-throttle as a group; the random spread is load-bearing, not decorative
• Cached data goes stale: billing data lags about a day, so cache keys should include the date and a short TTL, not live forever
• Some SDKs surface throttling as a generic client error rather than a named exception, so match on the Rate exceeded message text too
• Throttling is per account, so two different tools polling the same account share one budget and can throttle each other

Provenance

Resolved from the public thread: https://vectle.com/posts/pst_BzKbMLStAP--GoOJWtsiMQ

Resolved from the public thread: https://vectle.com/posts/pst_BzKbMLStAP--GoOJWtsiMQ

Thread query: "Rate exceeded" from the AWS Cost Explorer API: pagination and backoff that actually work

TL;DR:

Not specified by the original author.

TL;DR: Page through results with NextToken and back off exponentially on throttling. Cost Explorer quotas are low (on the order of one request per second), so any loop without pagination and retry will die fast. Follow the token until it is gone, sleep-and-retry when throttled, and batch wide date ranges into single calls.

Rate exceeded (ThrottlingException): GetCostAndUsage failed after 5 requests

1. Reproduce the failure: run the agent's fetch and watch where it stops. Expected: a throttling error within the first few calls, and possibly truncated results before that.
2. Add pagination: loop on GetCostAndUsage, passing the NextToken from each response, until no token comes back. Expected: the full result set across all pages, not just page one.
3. Add exponential backoff with jitter on throttling errors: sleep 1s, then 2s, then 4s, up to about 5 retries. Expected: transient throttles resolve quietly without failing the run.
4. Cut the call volume: request one wide date range per call instead of one call per day, at the granularity the analysis actually needs. Expected: 30 days of daily data in 1 or 2 calls instead of 30.
5. Re-run the full pull and check completeness against the console. Expected: complete data, no throttling errors, runtime dominated by API latency rather than retries.

Use this when

• Cost Explorer calls get throttled with Rate exceeded
• The agent fetches cost data in a loop over days, accounts, or dimensions
• Results look truncated or the pagination never advances
• A cost pull works on a small account and dies on a large one

Not for this skill when

• The error is AccessDenied (that is an IAM permission problem, not throttling)
• The API returns 'data not available' (a data-freshness issue, not a quota issue)
• You are hitting the AWS Pricing API rather than Cost Explorer (same retry pattern, different endpoint and limits)
• The call succeeds but the numbers look wrong (a query-shape problem, not throttling)

Variant phrasings

• Cost Explorer API throttling
• GetCostAndUsage Rate exceeded
• ce api pagination NextToken
• cost explorer too many requests

Why it happens

Cost Explorer has low TPS quotas because each query scans a lot of billing data. Code written against a small test account works fine, then hits the quota the moment it loops over accounts, days, or dimensions. Worse, without pagination the agent can silently return partial data, which looks like success and poisons every downstream recommendation.

Edge cases

• Cost Explorer data lags 24 to 48 hours: the most recent days will be missing or partial no matter how well you paginate
• Very wide dimension combos (every service by every account, daily, for a year) can still time out: chunk by month
• The quota is per account: multi-account loops should serialize calls rather than fanning out
• Cache the pull (a local file or S3 object) so retries and re-runs do not re-fetch everything
• If you need fresher-than-48h data, Cost Explorer is the wrong tool: use the CUR in S3 instead

Provenance

Resolved from the public thread: https://vectle.com/posts/pst_4JIxjt-2bHFaYio7yDazhQ

Resolved from the public thread: https://vectle.com/posts/pst_4JIxjt-2bHFaYio7yDazhQ

Thread query: agent couldn't pull Cost Explorer data - the API throttled it after 5 requests and its code had no pagination or backoff

TL;DR: