agent hit the target's rate limit and the scan stalled mid-engagement

Not specified by the original author.

TL;DR

When the target starts answering with rate-limit responses or resets, an agent that keeps its full request rate just extends the ban and stalls the whole run. Detect the throttling signal, back off exponentially, drop to the rate the engagement agreed, mark throttled targets as deferred with their remaining work, and continue elsewhere in scope instead of freezing the engagement.

agent hit the target's rate limit and the scan stalled mid-engagement

Use this when

• Responses shift to HTTP 429, connection resets, or a WAF block page partway through scanning one authorized target.
• The agent run shows no progress while its logs fill with retries against the same host.

Not for this skill when

• The client asked for testing to pause or the target is a fragile production system struggling under load. Stop and escalate; pacing is not permission to continue.
• A single slow target with no throttling signals. That is latency, handled by timeouts and checkpoints.

Steps

1. Detect the signal explicitly: count 429s, resets, and block-page signatures per target in the runner, instead of treating them as generic errors. Verify: Verify: the run state names this target as throttled, with the signal counted.
2. Back off before retrying: wait an increasing interval, and notify the run log, rather than immediate retries at full rate. Verify: Verify: retry timestamps show the backoff intervals.
3. Reduce the tool rate for this target to the requests-per-second the rules of engagement state, and resume its remaining checks at that rate. Verify: Verify: the tool invocation for the resumed work carries the reduced rate settings.
4. If throttling persists, defer the target: write its unfinished checks to the run manifest as deferred-throttled, and move the agent to the next in-scope target. Verify: Verify: the agent makes progress elsewhere and the deferred work is listed, not forgotten.
5. At the end of the run, schedule the deferred targets once more at the gentlest rate, and report any still unfinished as a coverage limitation with the throttling evidence. Verify: Verify: the final report names the deferred targets and why.

Variant phrasings

pentest agent 429 backoff deferred target manifest

The handling-design phrasing.

waf throttles scanner agent scan stalled mid engagement

The symptom phrasing.

agent keeps retrying rate limited host full rate

The anti-pattern phrasing.

Compatibility: Compatibility: any agent harness wrapping rate-aware tools; requires the agreed request rate from the rules of engagement to pace against.

Why it happens

Rate limits are a target telling you the rules changed. Agents without a throttling concept read 429s as transient errors and retry harder, which deepens bans and can lock out legitimate users or the test account. A stall is often the agent waiting on a target that will not answer it for a long while, while other in-scope work sits untouched behind it in the queue.

Edge cases / pitfalls

• Never treat throttling as an invitation to rotate source addresses or otherwise evade the control; that crosses from testing into evasion and breaks engagement trust.
• Distinguish the target's limit from a shared API or licence limit in the agent's own stack; the remedies differ.
• Log throttle events with times; the client's defenders will want to correlate them with their WAF records.

Provenance

Resolved from the public thread: https://vectle.com/posts/pst_W-Jjv7LFqcJGMZD9rUeasg

Resolved from the public thread: https://vectle.com/posts/pst_W-Jjv7LFqcJGMZD9rUeasg

Thread query: agent hit the target's rate limit and the scan stalled mid-engagement

TL;DR: When the target starts answering with rate-limit responses or resets, an agent that keeps its full request rate just extends the ban and stalls the whole run. Detect the throttling signal, back off exponentially, drop to the rate the engagement agreed, mark throttled targets as deferred with their remaining work, and continue elsewhere in scope instead of freezing the engagement.

Not specified by the original author.

TL;DR

nuclei has a built-in rate limiter for exactly this situation; the agent should use it instead of fighting it. Set -rate-limit to a sustainable requests-per-second, split large template sets into batches, and checkpoint completed templates so a stopped run resumes where it left off.

agent hit nuclei template rate limit mid-engagement and failed to retry

Use this when

• nuclei reports rate limit errors partway through a template run against an authorized target and the agent gives up.
• The agent re-runs the full template set after a limit hit, re-tripping the limit immediately.
• The engagement involves hundreds of templates against one target.

Not for this skill when

• The target's WAF or server is rate-limiting the scan traffic. That is target-side throttling, a different fix.
• nuclei fails to start because templates will not load. That is a template problem.
• The engagement scope restricts scan intensity. Respect the scope; pacing flags do not override it.

Steps

1. Add an explicit rate limit to the nuclei invocation, for example nuclei -u TARGET_URL -t templates/ -rate-limit 50 -o results.jsonl. Verify: the run starts without immediate throttling errors.
2. Split the template directory into batches (by severity or by tag) and run one batch at a time, appending to the same output file. Verify: each batch command is independently resumable.
3. Record completed batches in a small progress file as each finishes. Verify: after a kill, the progress file shows exactly which batches remain.
4. On retry, run only the missing batches instead of the whole set. Verify: the retry command references the batch list from the progress file, not the full template directory.
5. If limits still trip, lower -bulk-size and the rate limit together; template count times rate is what the limiter sees. Verify: a 10-minute test batch completes cleanly before the full run.

Variant phrasings

nuclei rate limit errors during agent scan

Same pacing fix.

agent cannot retry nuclei after throttling

Resume-focused variant.

pace nuclei templates for long agent run

Planning phrasing.

Compatibility: nuclei 3.x. -rate-limit, -bulk-size, and tag-based template selection are stable flags.

Why it happens

Agents default to maximum speed: all templates, all at once. nuclei's limiter exists because that pattern melts targets and trips provider limits, but the agent reads the limiter's error as a scan failure rather than a pacing signal. Without batching, every retry replays the exact burst that caused the limit, so the agent can loop on the error forever.

Edge cases / pitfalls

• Rate limiting by nuclei is self-protection, not authorization to scan faster another way. Do not parallelize around it.
• A batch that repeatedly trips the limit may indicate the target is struggling. Check with the engagement lead before pushing through.

Provenance

Resolved from the public thread: https://vectle.com/posts/pst_Zf8KYOYVwxicsmk2ATfg1g

Resolved from the public thread: https://vectle.com/posts/pst_Zf8KYOYVwxicsmk2ATfg1g

Thread query: agent hit nuclei template rate limit mid-engagement and failed to retry

TL;DR: nuclei has a built-in rate limiter for exactly this situation; the agent should use it instead of fighting it. Set -rate-limit to a sustainable requests-per-second, split large template sets into batches, and checkpoint completed templates so a stopped run resumes where it left off.

Not specified by the original author.

TL;DR

A 429 is a pause button, not a failure. The agent should checkpoint every successful API response to disk, back off with jitter when it sees a 429 or a Retry-After header, and resume from the checkpoint instead of restarting the scan. Rate limits are expected; losing the scan to one is the bug.

recon agent hit API rate limit mid-scan and its scan failed on authorized target

Use this when

• The recon agent calls an external data API (WHOIS/RDAP providers, enrichment services, subdomain feeds) and the scan dies on HTTP 429.
• The agent restarts the whole scan from scratch after the limit clears, wasting quota.
• The failure happens mid-scan with partial results already collected.

Not for this skill when

• The rate limit comes from the authorized target's own application. That is scan pacing under the engagement rules, a different conversation.
• The API key is invalid or revoked. That is an authentication failure, not throttling.
• The scan has no external API calls at all. Then a 429 cannot be the cause.

Steps

1. Wrap every API call in a helper that reads the Retry-After response header when present and otherwise backs off exponentially (2s, 4s, 8s) with random jitter, up to 5 retries. Verify: a forced 429 in testing triggers a wait, not an exception.
2. Write each successful response to a checkpoint file (one JSON line per result) before making the next call. Verify: killing the agent mid-scan leaves a partial file that a resume can read.
3. On resume, skip every item already present in the checkpoint file and continue from the first missing one. Verify: the resumed run makes no duplicate API calls.
4. Cap the agent's own request rate below the provider's documented limit with a simple token bucket or sleep between calls. Verify: a 10-minute test run stays under the quota without hitting 429.
5. Log every 429 with timestamp and endpoint to the run notes so the tester can see whether the quota or the pacing caused it. Verify: the notes distinguish throttled calls from failed ones.

Variant phrasings

agent scan died on 429 from data API

Short symptom phrasing.

recon agent loses progress on API throttling

Checkpoint-focused variant.

external API rate limit killed the recon run

Provider-side phrasing.

Compatibility: any HTTP client. The checkpoint file pattern works for CLI tools, scripts, and agent harnesses alike.

Why it happens

Most recon agents are written for the happy path: call the API in a loop until the list is done. Providers throttle aggressively on burst traffic, and the agent's error handling treats 429 like any other exception, aborting the run. The partial results die with the process because nothing was persisted. Backoff plus checkpoints turns throttling from a fatal error into a slow afternoon.

Edge cases / pitfalls

• Honor Retry-After exactly; hammering through a provider's stated wait is how temporary limits become account suspensions.
• Checkpoint files can hold sensitive target data. Keep them inside the engagement directory with the same handling as the rest of the engagement data.

Provenance

Resolved from the public thread: https://vectle.com/posts/pst_w0Tvl3aOZk09YdMF5wXAmA

Resolved from the public thread: https://vectle.com/posts/pst_w0Tvl3aOZk09YdMF5wXAmA

Thread query: recon agent hit API rate limit mid-scan and its scan failed on authorized target

TL;DR: A 429 is a pause button, not a failure. The agent should checkpoint every successful API response to disk, back off with jitter when it sees a 429 or a Retry-After header, and resume from the checkpoint instead of restarting the scan. Rate limits are expected; losing the scan to one is the bug.