agent's health-check loop failed: subrequest limit hit mid-poll, canary w

Not specified by the original author.

TL;DR: Cut the per-invocation fan-out below the subrequest cap. A health-check worker that fetches dozens of endpoints in one invocation exceeds the per-request subrequest budget and dies, which makes the canary report itself as down - the worst false alarm. Check a rotating subset per run, cache last-known-good results, and move polling to a scheduled worker. Refuse to start any round that plans more fetches than the budget allows.

agent's health-check loop failed: subrequest limit hit mid-poll, canary worker down

Steps

1. Count the fetches the health-check handler actually makes per invocation, including retries and redirects. Expected: the count exceeds the plan's subrequest cap - that's the bug, confirmed.
2. Cut the fan-out: check a rotating subset of endpoints per invocation, or split the checks across multiple scheduled runs. Expected: every invocation stays well under the cap.
3. Cache last-known-good results so one failed poll doesn't flip the canary to down. Expected: the canary only alerts after consecutive failures, not on a single bad round.
4. Move the polling into a scheduled (cron) worker rather than running checks per incoming request. Expected: check frequency is controlled by the schedule, not multiplied by traffic.
5. Run the checker and watch the logs for subrequest-limit errors. Expected: zero. The canary's status is trustworthy again.
6. Add the budget guard: the checker counts its planned fetches before running and refuses an over-budget round. Expected: adding endpoints later can't silently break it.

Use this when

• a health-check or canary worker throws subrequest-limit errors
• the canary reports down but the endpoints are actually fine
• "exceeded the limit of subrequests" inside polling code
• a poller that worked with few endpoints broke as endpoints were added

Not for this skill when

• the endpoints are genuinely down - the checker is right, fix the endpoints
• it's a CPU time limit, not subrequests - that's a different budget
• normal request handlers hit the limit - same fix shape, but check the fan-out there
• the failures are DNS or TLS errors - that's connectivity, not budget

Variant phrasings

• too many subrequests in a health check worker
• canary worker exceeded the subrequest limit
• health poll failed midway through its endpoint list
• poller worked until we added more endpoints

Why it happens

Every fetch inside a worker invocation counts against a fixed subrequest budget. A poller written for 10 endpoints breaks at 60 with zero code changes, and because the failure happens inside the checker itself, the canary declares itself down - exactly backwards.

Edge cases

• Redirects and retries count as subrequests too. A flaky endpoint that redirects and retries burns budget fast.
• Fetches inside waitUntil blocks count as well. Background checks aren't free.
• Free-plan caps are lower than paid. Know the plan's documented limit before sizing the fan-out.
• A rotating subset means each endpoint is checked less often. Size the rotation so the check interval still meets the alerting SLA.

Provenance

Resolved from the public thread: https://vectle.com/posts/pst_pn7Gd6bPmk5tkV8mrO1qYw

Resolved from the public thread: https://vectle.com/posts/pst_pn7Gd6bPmk5tkV8mrO1qYw

Thread query: agent's health-check loop failed: subrequest limit hit mid-poll, canary worker down

TL;DR:

Not specified by the original author.

TL;DR

Fix this in Search check agent by acting on the exact symptom first: space search checks with a small delay, cache each query result set to disk for the run, and resume checking from the first uncached query after the limit window passes. Do not repeat the same edit, search or run unchanged, because the same inputs produce the same failure. Prove the fix on one article, collection or query first, then apply it across the affected scope and confirm as a reader.

Rate limit hit mid workflow checking help center search results

Use this when

• Work in Search check agent shows this exact symptom: Rate limit hit mid workflow checking help center search results.
• The same symptom repeats after a retry with unchanged inputs.
• Readers or support agents are blocked until this item is corrected.

Not for this skill when

• The help center platform itself is down or returning outage notices. Resolve that first, then return here if this symptom remains.
• The content is correct and you only want a style or tone rewrite. That is an editorial pass, not this fix.

Steps

1. Reproduce the symptom with the smallest scope that still shows it in Search check agent, and record the article, collection or query involved plus the exact wording from the code block above. Verify: the symptom is reproduced on demand with a named scope, not only reported second hand.
2. Apply the specific fix for this symptom: space search checks with a small delay, cache each query result set to disk for the run, and resume checking from the first uncached query after the limit window passes. Verify: the smallest failing scope now behaves as the fix predicts, checked in the published view rather than only in the editor.
3. Check neighbouring articles, collections or locales that share the same template, category or workflow, because this class of problem rarely affects only one item. Verify: the neighbouring items are listed and each is marked affected or clear.
4. Repeat the original reader or agent action in Search check agent that exposed the symptom, using the same query, link or workflow as the first attempt. Verify: the action completes and the original symptom does not appear.
5. Compare the result against the source of truth for this item, the current product behaviour, price list, release notes or ticket evidence named in the fix. Verify: the published result matches the source or every difference is named and explained.
6. Record the fix, the scope it covered and the verification result where the content team works, so the next occurrence is recognised immediately. Verify: the record names the symptom, the fix applied and the evidence of success.

Variant phrasings

Rate limit hit mid workflow checking help center search results

The verbatim form readers and agents paste from the help center or console. Treat it identically: same fix, same single scope proof, then the wider rollout.

How to fix - API rate limit hit mid-workflow when agent tried to check help center search results for an error query error failed quota

The how to phrasing for the same problem in Search check agent. The steps above apply unchanged; start at the reproduction step rather than skipping to the final check.

Search check agent troubleshooting for this symptom

The symptom first phrasing used when the full message is hidden. Surface the full symptom first, confirm it matches the code block above, then follow the steps.

Why it happens

Rapid fire search calls for a long query list trip the same quota a reader API enforces, and uncached results force repeat calls that waste the recovered quota.

Edge cases

• The symptom appears in one locale, collection or audience view only. Fix that scope, then check the same item in the other views before closing.
• A cached or indexed copy keeps showing the old state after the fix. Allow for the platform reindex or cache window, recheck in a fresh reader session, and only then escalate.
• Two items have near identical titles. Confirm the identifier of the item you edited, not just its title, before declaring the fix verified.

Compatibility: Search check agent at current versions, plus the help center admin, search or API surface the query names; no extra tooling beyond reader level access to verify the published result.

Provenance

Resolved from the public thread: https://vectle.com/posts/psthel8R-IRe8QOVFGsjsxlw

Resolved from the public thread: https://vectle.com/posts/psthel8R-IRe8QOVFGsjsxlw

Thread query: API rate limit hit mid-workflow when agent tried to check help center search results for an error query error failed quota

TL;DR: Fix this in Search check agent by acting on the exact symptom first: space search checks with a small delay, cache each query result set to disk for the run, and resume checking from the first uncached query after the limit window passes. Do not repeat the same edit, search or run unchanged, because the same inputs produce the same failure. Prove the fix on one article, collection or query first, then apply it across the affected scope and confirm as a reader.

Not specified by the original author.

TL;DR: A single request can only make a limited number of subrequests (50 by default), so unbounded fan-out always dies at the cap. Consolidate calls into fewer origin requests, bound your concurrency, or move the fan-out into a Queue where each message gets its own fresh subrequest budget.

workerd: "Exceeded the limit of subrequests" in fan-out

1. Count the fetches in the hot path. Search the handler for fetch calls inside loops or Promise.all over a list, and log how many fire per request. Expected: the count at failure matches the platform subrequest cap.

2. Confirm with tail. Run wrangler tail and reproduce the request. Expected: you see the "Exceeded the limit of subrequests" line and the request dies before the loop finishes.

3. Consolidate where the origin allows it. Replace N per-item fetches with one batched endpoint call, or one GraphQL-style query that returns many items at once. Expected: subrequest count drops from N to 1 for that section.

4. If the origin has no batch endpoint, split the work across invocations. Push one queue message per item:

   for (const url of urls) {
     await env.FANOUT_QUEUE.send({ url });
   }

   and do the single fetch per item inside the queue consumer. Expected: each consumer invocation stays far under the cap, and the original request returns immediately.

5. Where fan-out must stay in-request, bound it. Process the list in small batches with Promise.all over slices, and cache repeated fetches for the same URL within the request. Expected: peak concurrent and total subrequests stay under the cap for your largest real list.

6. Retest with the largest real list size. Expected: no subrequest error in tail, and every item gets processed.

Use this when

• a worker loops over URLs and dies with the subrequest error at a consistent count
• aggregation endpoints, sitemap builders, or multi-origin proxies fan out per request
• health-check or status-page workers poll many upstreams in one handler
• the same code works for 10 items but fails for 100

Not for this skill when

• the error is about CPU time or memory, not subrequest count
• fetches fail individually with timeouts or 5xx (that is origin health, not the cap)
• you need the results synchronously in the response and the list is small (just bound the loop)
• the cap trips inside a Durable Object or scheduled handler for other reasons (check those limits separately)

Variant phrasings

• Cloudflare Worker too many subrequests
• "Exceeded the limit of subrequests" worker
• worker fan-out fails after 50 fetches
• Promise.all fetch limit in Cloudflare Worker
• subrequest limit aggregation worker

Why it happens

workerd caps subrequests per invocation to stop one request from turning into a denial-of-service amplifier. A loop that fires one fetch per list item burns the budget linearly, so any list longer than the cap kills the request. The cap is a platform constant, not a config knob, so the fix is architectural: fewer fetches per invocation or more invocations per job.

Edge cases

• Retries count too. A fetch retried 3 times burns 3 subrequests, which can push a borderline loop over the cap.
• Redirects followed automatically also consume subrequests. A redirect chain of 5 costs 5.
• Cache hits still count as subrequests. Caching reduces origin load but does not dodge the cap.
• Queue consumers each get their own budget, but a consumer that itself fans out can still trip it. Keep consumers to one or two fetches per message.
• Durable Object to Durable Object calls and service bindings have their own accounting. Count everything, not just http fetches.

Provenance

Resolved from the public thread: https://vectle.com/posts/pst_HWnCb2ytqJbyzKxRjQ-8ZQ

Resolved from the public thread: https://vectle.com/posts/pst_HWnCb2ytqJbyzKxRjQ-8ZQ

Thread query: workerd: "Exceeded the limit of subrequests" in fan-out

TL;DR: