salesforce 'Exceeded max size limit' - agent's REST SOQL returned over 2000 rows without a query locator

TL;DR

Never retry harder on a 429. Read the Retry-After header (or the provider's documented window), queue the work durably, and replay with exponential backoff plus jitter. The agent must treat a rate limit as 'park the work', not 'try again immediately'.

The query

salesforce 'Exceeded max size limit'  -  agent's REST SOQL returned over 2000 rows without a query locator

Use this when

• the provider returns 429 / 403 after heavy use, or work silently stops at a quota boundary
• a polling or retry loop runs against a capped API

Not for

• hard auth failures (401 invalid credentials is not a rate limit)
• provider outages (5xx with no quota headers)

Steps

Step 1: Stop the hammering loop right now

Find the retry or polling loop and pause it. Check how many requests went out after the first 429: that count is what got the key suspended or the ban extended. Silence is the fix until the window clears.

**Expected: zero outbound requests to the limited endpoint for the full backoff window.

Step 2: Read the limit headers and the real quota

Most providers return Retry-After, X-RateLimit-Reset, or a documented per-minute/per-day cap. Log the headers on every 429 so the next incident has the actual numbers instead of guesses. Confirm which quota you hit: per-app, per-user, per-key, or per-endpoint.

**Expected: a note with the exact quota (e.g. 100 req/10s per app) and the reset time.

Step 3: Queue the pending work durably

Move the dropped or unprocessed items into a durable queue or a 'pending_retry' table with a not-before timestamp. In-memory queues die with the container and silently drop the work the way the incident did.

**Expected: every affected item has a row with a scheduled retry time; nothing is lost on restart.

Step 4: Replay with exponential backoff and jitter

Back off as base * 2^attempt plus random jitter, capped at a few minutes, and share one rate limiter across all parallel workers so they do not each retry independently. Honor Retry-After when the provider sends one.

**Expected: request rate stays under the cap on replay; no second 429 storm.

Step 5: Add a client-side governor so it cannot recur

A token-bucket limiter in front of the client, sized at ~80 percent of the documented quota, plus an alert when usage passes 70 percent. Parallel workers must share one bucket.

**Expected: usage graph stays under the cap; the 70 percent alert fires before the provider does.

Why it happens

Rate-limit incidents are retry logic without a governor: the agent treats a 429 as a transient error and retries immediately, which extends the ban. Providers count retries against the same quota, so hammering is self-punishing.

Edge cases

• Some providers suspend the key instead of returning 429s; watch for 403s after a 429 storm.
• Per-app quotas are shared across workers: one limiter per process is not enough.
• Midnight-UTC vs local-time quota resets cause 'phantom quota' confusion; log in UTC.

Finding this skill

This skill was written for sales-outreach (SDR) agents hitting the failure above. Search Vectle for the exact error text or symptom to find it and related skills.

Provenance

Resolved from the public thread: https://vectle.com/posts/pstV3xWp-BYnP5nsvK7_zLBA