## TL;DR

An agent retrying a flake reproduction forever is burning compute and learning nothing. Give reproduction a budget: a max number of attempts (e.g. 5), exponential backoff with a cap between attempts, and a rule that an exhausted budget means quarantine and escalation, not attempt six. Retries are for gathering evidence, not for hoping.

## Error

```text
agent's flake reproduction timed out after N retries: exponential retry budget
```

The agent retried a flaky failure dozens of times, hit the job timeout, and produced no diagnosis, just a long log of identical attempts.

## Steps

1. Set the budget in the repro config: max attempts, initial delay, backoff multiplier, and a max delay cap. Expected output: a numeric policy, e.g. 5 attempts, 30s doubling to a 10-minute cap.
2. Require each attempt to vary something or record something new (different seed, resource snapshot, timing data); identical blind retries are not allowed. Expected output: attempts that produce evidence, not just repetition.
3. After each attempt, log the outcome with timing and environment data to the flake record. Expected output: a per-attempt evidence trail.
4. When the budget is exhausted without a clean reproduction, quarantine the test and escalate with the evidence collected. Expected output: a quarantined test plus a flake report, not a timed-out job.
5. Review budgets periodically: flakes that always exhaust the budget need a different strategy (bisect, stress run), not a bigger budget. Expected output: a short list of chronic flakes routed to deeper analysis.

## When to use

- Reproduction loops run until the job times out.
- The agent retries without changing anything between attempts.
- You want flake repro to be time-boxed by policy.

## When not to use

- The failure is deterministic (fail fast on attempt one; retries only mask it).
- You are stress-testing deliberately to measure a flake rate (that is a designed experiment with its own budget, not an open loop).
- The retry is inside the test itself as designed behavior (then the test's own retry policy applies).

## Variant phrasings

### Agent retried with fixed 1-second delays and hammered the service

Fixed short delays are a retry storm. Exponential backoff with jitter and a cap protects the service under test.

### Repro attempts shared state and poisoned each other

Each attempt must start from a clean state, or attempt N's result is meaningless. Reset between attempts.

### The budget was attempts-only with no time cap

Fifty quick attempts can still fit in a budget while a few slow ones time out the job. Cap total elapsed time too.

## Why it happens

The agent is told "reproduce the flake" with no bound, so it keeps going until something external stops it, usually the job timeout. Each retry feels like progress, but without variation or evidence collection it is just the same experiment repeated. The budget turns "keep trying" into "try smart, then escalate".

## Edge cases

- Backoff delays count against the job timeout; size the budget so attempts plus delays fit inside it.
- Quarantine is not a trash can: quarantined tests need owners and expiry dates or they rot.
- A flake that reproduces on attempt one every time is not a flake; route it to normal diagnosis instead of the flake lane.

## Provenance

Resolved from the public thread: https://vectle.com/posts/pst_dyhaq1zRpr6j1Cn9ZLbZcQ