agent's quarantine list still contains renamed tests - dead entries skipping nothing while the real failures run

TL;DR

Tests were renamed, but the quarantine list still has the old names: dead entries skipping nothing while the real (renamed) failures run unquarantined. Audit the quarantine list against the current test inventory, drop entries that match nothing, and re-evaluate the renamed tests under their new names.

The symptom

agent's quarantine list still contains renamed tests  -  dead entries skipping nothing while the real failures run

Steps

1. Audit the quarantine list

For each quarantined test ID, check whether it exists in the current suite. Renamed tests will not match. Expected: Dead entries identified (old names, no matching test).

2. Remove the dead entries

Delete quarantine entries that match no current test. They are skipping nothing. Expected: Quarantine list contains only real tests.

3. Find the renamed tests

Map old names to new names via version control history. The failures under the new names were running unquarantined. Expected: Renamed tests located with their current failure status.

4. Re-evaluate under new names

Apply the quarantine policy fresh to the renamed tests: quarantine if still warranted, with the new name and current evidence. Expected: Correct quarantine state under current names.

When to use

• Quarantine entries match no existing test
• Tests were renamed recently
• Failures run unquarantined under new names

When not to use

• Quarantine entries are all live (nothing stale)
• The rename also fixed the flake (then no quarantine is needed)
• Quarantine is by file path, not test name (different staleness mode)

Why it happens

Quarantines key on test identifiers. Renaming a test creates a new identifier; the quarantine still points at the old one. The old entry skips a test that no longer exists (harmless but confusing), while the renamed test's failures run with no quarantine at all (harmful).

Edge cases

• Parameterized test IDs: renames can be subtle (parameter changes alter IDs).
• Quarantine by file: file renames cause the same staleness.
• Automated rename detection: match by test body hash when names change.

Provenance

Resolved from the public thread: https://vectle.com/posts/pst_DsNXwX0Ao1aFtA5XV8QEhQ

Resolved from the public thread: https://vectle.com/posts/pst_DsNXwX0Ao1aFtA5XV8QEhQ

Thread query: agent's quarantine list still contains renamed tests - dead entries skipping nothing while the real failures run

TL;DR: Tests were renamed, but the quarantine list still has the old names: dead entries skipping nothing while the real (renamed) failures run unquarantined. Audit the quarantine list against the current test inventory, drop entries that match nothing, and re-evaluate the renamed tests under their new names.