# Fix the locale QA agent loop re-running a failed screenshot suite

## TL;DR

The agent retries the whole screenshot suite on any failure, so quarantine the failing specs and continue. A single flaky screenshot retriggers everything, which looks like a loop and burns hours. Mark failed specs as quarantined after one retry, finish the suite, and report the quarantine list.

## The error

```text
Locale QA agent: screenshot suite run 1 failed, run 2 failed, run 3 failed...
same 2 specs fail each time, the suite never completes
```

## Fix it

### Step 1: Identify the consistently failing specs

```bash
grep -n "failed" logs/qa-run-*.log | sort | uniq -c | sort -rn | head -5
```

Expected: You see the 2 specs that always fail.

### Step 2: Quarantine after one retry

```bash
node -e "console.log('pattern: spec fails, retry once, on second failure mark quarantined and continue the suite')"
```

Expected: Flaky specs stop restarting the suite.

### Step 3: Re-run the full suite

```bash
node scripts/qa-screenshots.js | tail -3
```

Expected: The suite completes with a quarantine report.

### Step 4: Fix or accept the quarantined specs

```bash
node -e "console.log('triage each quarantined spec: real bug, flaky baseline, or obsolete -- then unquarantine')"
```

Expected: The quarantine list shrinks deliberately.

## When to use this

- A locale QA agent loops re-running a failed screenshot suite
- The same specs fail every run

## When NOT to use this

- Different specs fail each run, that is systemic flakiness
- The suite passes on retry, raise the flake tolerance instead

## Tool and version compatibility

- Screenshot QA suites with retry logic
- Quarantine patterns for flaky specs

## Variant phrasings

### quarantine list grows forever

Triage it weekly. Quarantine is a pause button, not a trash can.

### failures are real bugs not flakes

Then the loop is correct behavior begging for a fix. Fix the bugs, the loop stops.

## Why it happens

Suite-level retry treats any spec failure as a reason to redo everything. Two deterministically failing specs then guarantee infinite reruns, because the rerun fails the same way every time.

## Edge cases

- One retry distinguishes flakes from deterministic failures, keep it
- Report quarantined specs loudly, silent quarantine hides regressions
- Time-box the whole suite too, quarantine handles specs not hangs

## Provenance

Resolved from the public thread: https://vectle.com/posts/pst_quueZXPbIPbEqIwNuAzgWg