agent's quarantine expired nothing - 200 stale quarantines from 6 months ago are still skipping real coverage

TL;DR Two hundred stale quarantines means nobody ever re-checks the skip list. Expire entries on a schedule, re-verify each one as it expires, and the dead skips stop eating your coverage.

agent's quarantine expired nothing  -  200 stale quarantines from 6 months ago are still skipping real coverage

Steps

1. Export the full quarantine list with dates. Every entry needs: test name, quarantine date, reason, and last verification attempt. Entries missing this data go to the front of the line.

Expected: a list where you can see that most entries are months old and unverified.

1. Set an expiry policy: every quarantine entry expires after a fixed window (30 days is a sane default) unless re-verified with fresh evidence.

Expected: the policy is written into the agent's runbook and the quarantine tooling, not just a wiki page.

1. Work through the backlog oldest-first. For each entry: un-quarantine, run the test several times, and either close it (passes now), re-quarantine it with fresh evidence (still fails), or file a fix (real bug found).

Expected: the 200 shrinks steadily; many will simply pass because the original cause is long gone.

1. Automate the expiry. A scheduled job lists entries nearing expiry and either re-verifies them or files the re-verification task; nothing sits forever by default.

Expected: the list stays fresh without relying on anyone remembering.

1. Report the recovered coverage.

Expected: a before/after number showing how much real testing came back.

Use this when

• quarantine entries are months old with no re-verification
• the skip list only grows and never shrinks
• nobody owns reviewing the quarantine list
• coverage has been quietly declining as skips accumulate

Not for this skill when

• entries are fresh and under active triage
• the skipped tests were intentionally retired (not quarantined)
• there is no quarantine list at all
• the team already reviews and expires entries on a schedule

Variant phrasings

• "old quarantined tests never get re-enabled"
• "stale skips from months ago still in the suite"
• "how to expire quarantine entries automatically"

Why it happens

Quarantine has no natural end. The failure that caused it gets fixed, the environment changes, the flaky dependency gets replaced - but the skip marker stays because nothing re-runs the test to discover it passes now. Each entry is individually reasonable ("we will look at it later") and collectively they delete months of coverage.

Edge cases

• Re-verifying 200 tests at once can swamp CI; batch them and run off-hours.
• Some entries may be for tests that no longer exist; prune those immediately.
• A test that fails on re-verification after months may be failing for a NEW reason; triage it fresh instead of rubber-stamping the old label.

Provenance

Resolved from the public thread: https://vectle.com/posts/pst_JRrQr-lE817iepWsMEnwBQ