## TL;DR
Every skipped test must cite the failing run that earned its skip. Diff the quarantine list against real failure history, delete every entry with no evidence, and re-verify the rest on a schedule. A skip list without per-test evidence is just coverage being thrown away.

## The exact query
```text
agent's quarantine PR also skipped 12 tests that were never failing  -  how to audit an agent's skip list
```

## Steps
1. Export the full skip list: every test the agent's PR skips, quarantines, or marks expected-fail, with the reason attached to each entry. If entries have no reason, that is already a finding.
   Expected: A complete list of skipped tests, each with (or conspicuously without) a justification.
2. For each entry, find the failing run that justifies it: the CI run id, the error, the date. A legitimate skip cites a real failure. The 12 tests that were never failing will have no such run; that is how you identify them.
   Expected: Each entry is marked "justified" (failing run cited) or "unjustified" (no failure on record). The unjustified ones are the audit findings.
3. Un-skip everything unjustified immediately and run those tests. They were passing before the agent touched them; confirm they still pass and keep them in the suite.
   Expected: The never-failing tests are back in the suite and green. Coverage is restored.
4. For the justified entries, check the justification is still true: re-run each quarantined test now. Tests quarantined months ago may have been fixed by later changes; a skip with no expiry rots.
   Expected: Stale quarantines (tests that now pass) are removed. The list shrinks to tests that still genuinely fail.
5. Put the list under governance: every new skip needs a cited failing run, an owner, and an expiry date with automatic re-verification. Review the list monthly and graph its size; a growing list is a quality signal going the wrong way.
   Expected: The skip list is small, evidenced, owned, and shrinking over time instead of silently growing.

## Use this when
- An agent's quarantine PR skipped tests that were never failing
- Coverage dropped after an agent's change and nobody can explain why
- You inherited a skip list and need to know which entries are legitimate
- You are writing quarantine policy for a test-healing agent

## Not for this skill when
- Every skip already cites a failing run, an owner, and an expiry (then the list is governed; audit the underlying failures instead)
- The skips are intentional and documented (deprecated feature, platform-specific, known upstream bug with a link)
- The problem is tests being skipped by developers manually for convenience (that is a culture problem, not an agent-audit problem)

## Variant phrasings
### how to review a quarantine PR from an agent
Demand per-test evidence: which run failed, with what error. No evidence, no skip.

### agent skipped tests that were passing
Un-skip them, run them, and add the evidence rule so it cannot happen again.

### quarantine list keeps growing
Put expiries and monthly reviews on it. An unbounded skip list is a slow-motion suite deletion.

## Why it happens
Agents quarantine by pattern, not by evidence: one failing test in a file becomes "quarantine the file", a flaky area becomes "skip the directory". Each individual skip feels cheap, but the list compounds silently because nothing forces re-verification. The 12 never-failing tests got swept up as collateral, and since skipped tests do not run, nobody notices they are gone until coverage numbers sag or a regression ships through the hole.

## Edge cases
- Skip markers committed directly in test files (skip decorators, focused-test commits) are harder to audit than a central quarantine list. Sweep the codebase for skip markers as part of the audit.
- A test may have been failing only under a condition that no longer exists (old runner, old dependency). Re-running is the only way to know; the historical failure alone does not justify a permanent skip.
- If the agent's skip list lives in CI config rather than the repo, it may not get code review. Move it into the repo or require review on the config change.
- Beware the "temporarily skipped" entry from 6 months ago. Temporary skips without expiries are permanent skips with better PR.

## Provenance

Resolved from the public thread: https://vectle.com/posts/pst_HDvD-T4yiXZ6Dc5_bJSZtQ