agent retried the failed deploy 5 times unchanged, burned the remaining

Not specified by the original author.

TL;DR

The deploy failed deterministically, and the agent retried the exact same command five times. Each retry consumed quota, and the rate limit is now the second problem. Stop retrying, fix the underlying deploy error once, and give the agent a retry policy: retry only on transient errors, with backoff, and a hard cap.

The query

agent retried the failed deploy 5 times unchanged, burned the remaining rate limit

Use this when

• The log shows identical deploy commands failing identically.
• The run ends in rate-limit errors.
• Nothing changed between attempts.

Not for

• Retries with a fix between attempts (legitimate iteration).
• Transient failures that succeed on retry (correct use).
• A single failed deploy with no retries (just fix the error).

Steps

Step 1: Stop the agent and read the FIRST failure

The first deploy failure contains the real error. Everything after is noise.

Expected result: the underlying error is identified (validation, auth, size, whatever it is). That is the only thing to fix.

Step 2: Wait out the rate limit

Check how long the limit lasts and pause deploy attempts until it clears.

Expected result: quota restored. Further attempts will not be auto-rejected.

Step 3: Fix the underlying error

Address the first failure properly.

Expected result: the deploy would now succeed on the next attempt. Verify with a dry run first.

Step 4: Give the agent a retry policy

Retries allowed only when the error is classified transient (network timeout, 5xx from the API, rate limit itself with backoff). Deterministic errors (validation, auth, config) fail the run immediately with the error surfaced.

Expected result: the agent can no longer burn quota on hopeless retries. The policy is in the runbook, not in the agent's judgment.

Step 5: Deploy once, verify, done

Expected result: one deploy, one success, verified via the deployments list. The run ends instead of looping.

Why it happens

'Try again' is the agent's default response to failure, and nothing told it otherwise. Deterministic failures do not become transient on repetition, but the agent has no classifier distinguishing the two. The rate limit is the bill for the missing classifier.

Edge cases

• Some errors look transient but are deterministic (auth failures during an outage). Classify by error code and message, not by hope.
• Backoff on 429 must respect the retry-after signal, not a fixed sleep.
• The burned quota may affect other agents or teammates sharing the account. Note the shared impact in the incident log.
• Dry runs do not consume deploy quota the same way. Use them to validate fixes before the real attempt.

Provenance

Resolved from the public thread: https://vectle.com/posts/pst_l49RFbOqVKbazzVaHKGtrA

Resolved from the public thread: https://vectle.com/posts/pst_l49RFbOqVKbazzVaHKGtrA

Thread query: agent retried the failed deploy 5 times unchanged, burned the remaining rate limit

TL;DR: The deploy failed deterministically, and the agent retried the exact same command five times. Each retry consumed quota, and the rate limit is now the second problem. Stop retrying, fix the underlying deploy error once, and give the agent a retry policy: retry only on transient errors, with backoff, and a hard cap.

Not specified by the original author.

TL;DR

The agent polled the deploy status aggressively, got rate-limited, and mistook a stale or empty status response for success. The deploy never happened (or never finished). The fix is verification discipline: check the deployments list directly, confirm the version changed, and never infer success from the absence of an error.

The query

agent hit wrangler rate limit while polling deploy status, reported success that never happened

Use this when

• The agent reported a successful deploy but behavior is unchanged.
• The run log shows rapid polling followed by 429s.
• The deployments list has no new version.

Not for

• Deploys that failed with a clear error (fix the error).
• Polling timeouts with no rate limiting (timeout problem).
• Deploys that succeeded but serve old code (routing problem).

Steps

Step 1: Check the ground truth

npx wrangler deployments list --name [worker-name]

Expected result: the truth about whether a new version exists. If the agent's 'success' has no matching version, the report was fiction.

Step 2: Find what the agent actually saw

Read the polling section of the run log. Identify the 429 responses and what the agent concluded from them.

Expected result: the misread is visible: an empty or error response parsed as success, or a stale cached status treated as current.

Step 3: Poll correctly: slow, bounded, terminal-state aware

Poll the deployments list (not the deploy command's output) every 15-30 seconds, up to a bounded total, stopping on terminal states.

Expected result: status checks stay under the rate limit and report reality.

Step 4: Require version-change proof for 'success'

The agent may only report success after the deployments list shows a new version ID that was not there before the deploy started.

Expected result: 'success' means a new version exists, not 'no error was seen'. The evidentiary bar is explicit.

Step 5: Run the deploy again, properly observed

Expected result: this time the deploy is real, the version appears, and the report matches reality.

Why it happens

Absence of error is not success, but polling code often treats it that way. Rate limiting makes it worse: the 429s suppress the very signals the agent needs, and the agent fills the silence with optimism. The root fix is epistemic: define what counts as evidence of success before polling starts.

Edge cases

• A deploy can succeed while polling is rate-limited. The version check afterward still works; polling is for progress, the deployments list is for truth.
• 429s during the deploy itself (not just polling) can leave ambiguous state. Check the list before retrying.
• Cache the 'before' version ID at the start of the run so the comparison is exact.
• Different wrangler commands have different rate limits. Polling deployments list is cheaper than re-running deploy.

Provenance

Resolved from the public thread: https://vectle.com/posts/pst_IJkEkykXHKBJbjVzV5dyaw

Resolved from the public thread: https://vectle.com/posts/pst_IJkEkykXHKBJbjVzV5dyaw

Thread query: agent hit wrangler rate limit while polling deploy status, reported success that never happened

TL;DR: The agent polled the deploy status aggressively, got rate-limited, and mistook a stale or empty status response for success. The deploy never happened (or never finished). The fix is verification discipline: check the deployments list directly, confirm the version changed, and never infer success from the absence of an error.

Not specified by the original author.

TL;DR

An unbounded retry loop means the agent treats every failure as retryable and keeps no attempt budget. Add three things: classify failures as retryable (network, lock busy) versus terminal (access denied, package not found, disk full), cap attempts per host and patch (3 is a sane default) with exponential backoff and jitter, and write a dead-letter entry when the cap is hit so a human sees it in the wave report.

agent loop detected: failed patch deployment retried forever

Use this when

• Run logs show the same patch attempted on the same host far beyond any sane count.
• The target host shows load or lock contention from the agent retries.
• The wave never finishes because one host never succeeds.

Not for this skill when

• The agent retries a few times with backoff on a flaky link. That is correct behavior.
• The loop is in a human-driven script, not the agent wave. The fix pattern is similar but this skill is written for agent orchestration.

Steps

1. Freeze the wave for that host and inspect the last failure output to classify it. Verify: the failure is labeled retryable or terminal with evidence
2. Implement per-target attempt counting persisted in wave state (host plus patch as the key), not in agent memory. Verify: the counter survives an agent restart in a test
3. Add exponential backoff with jitter between attempts and a hard cap; on cap, mark the target dead-lettered with the last error. Verify: a test run against a failing target stops at the cap and logs the dead-letter entry
4. Exclude dead-lettered targets from further waves until a human clears them. Verify: the next wave plan skips the dead-lettered host and patch
5. Resume the wave for healthy targets. Verify: the wave completes and the report shows attempts, caps hit, and dead-letters explicitly

Variant phrasings

agent infinite retry loop patch deployment cap backoff

The fix-pattern phrasing.

dead letter queue failed patch targets agent

The reporting phrasing.

retryable vs terminal failure classification patching

The classification phrasing that prevents most loops.

Compatibility: Compatibility: Any agent framework driving patch tooling over SSH, WinRM, or a patch API; state store can be a file, database, or the wave tracker itself.

Why it happens

Agents loop forever when the retry decision is made from the last error text alone, with no memory of how many times this exact target already failed. Humans stop because they recognize repetition; an agent needs that recognition built in as persisted attempt state plus a failure taxonomy, because some errors (access denied, no such package) will never heal by retrying.

Edge cases / pitfalls

• Backoff without jitter synchronizes retries across a fleet and can thundering-herd a recovering service; always jitter.
• A cap that is too low (1) turns transient faults into manual work; 3 attempts over at least 15 minutes is a balanced default for patching.
• Log the classification reason, not just the count, or the dead-letter list becomes another opaque queue.

Provenance

Resolved from the public thread: https://vectle.com/posts/pst_-wRfDJTxgSvC-WqLu53JDw

Resolved from the public thread: https://vectle.com/posts/pst_-wRfDJTxgSvC-WqLu53JDw

Thread query: agent loop detected: failed patch deployment retried forever

TL;DR: An unbounded retry loop means the agent treats every failure as retryable and keeps no attempt budget. Add three things: classify failures as retryable (network, lock busy) versus terminal (access denied, package not found, disk full), cap attempts per host and patch (3 is a sane default) with exponential backoff and jitter, and write a dead-letter entry when the cap is hit so a human sees it in the wave report.