agent diagnosed leaked state from a passing test's fixtures that only initialize on first use

TL;DR

A passing test can still leak state through its fixtures. Fixtures that initialize lazily on first use, global singletons, cached clients, one-time setup with no teardown, write process-global state the first time any test touches them, and no test ever cleans it up because no test fails. Audit fixtures that touch shared resources for missing teardown, regardless of pass or fail.

The query

agent diagnosed leaked state from a passing test's fixtures that only initialize on first use

Steps

1. Inventory the fixtures of the tests around the failure

List every fixture used by the tests that run before the failure: their scope, what they initialize, and whether they define any teardown. Pay special attention to session- and module-scoped fixtures.

Expected: a fixture inventory with scope and teardown status for each. The suspects are scoped fixtures with init but no teardown.

2. Flag lazy one-time initialization

Look for the lazy pattern: a global that is None until first use, a cached client created on first call, a "setup once" flag. These initialize on whichever test happens to run first and then persist for the whole worker.

Expected: at least one fixture (or helper it calls) matching the lazy-init pattern, touching shared state.

3. Reproduce the leak deliberately

Force the fixture's init path in isolation, then inspect the shared state it leaves behind. Compare against the dirty state seen in the failing run.

Expected: the forced init reproduces the exact dirty state from the failure. Leak confirmed at the fixture level.

4. Add teardown to the fixture

Give the fixture cleanup that runs at the end of its scope: close the client, clear the cache, reset the singleton. Lazy init must be paired with eager teardown.

Expected: the shared state is clean after the fixture's scope ends, and the original failure is gone.

Use this when

• Shared state is dirty but every test in the run passed
• Fixtures do lazy or one-time initialization of global state
• The failure depends on which test first triggers the fixture's init
• The agent chased test bodies and found nothing, because the leak is in fixture code

Not for this skill when

• The leak comes from a test body that writes state directly (then the test needs teardown, not the fixture)
• Fixtures already have teardown and it runs (the leak is elsewhere: check the teardown's correctness)
• The shared state is set by global conftest setup or CI config (intentional, not a leak)
• No fixture touches the dirty resource (widen the search to imports and library side effects)

Variant phrasings

a cached HTTP client keeps auth from the first test that used it

The cache is the lazy singleton. Clear it in the fixture's teardown or scope the client per test.

the first test to run determines the dirty state for everyone

Whichever test triggers lazy init first owns the leak's shape. That nondeterminism is the fingerprint of this pattern.

agent blamed the test that failed instead of the fixture everyone shares

Fixtures are invisible in failure logs. When the victim's code is clean, read its fixtures next.

Why it happens

The agent's mental model is test-centric: tests do things, fixtures just set things up. Lazy initialization breaks that model because the state mutation happens inside fixture code, triggered implicitly by whichever test runs first, and it happens exactly once, so it never shows up as a repeated action in logs. The agent reads the failing test, reads the passing tests around it, finds nothing that writes the dirty state, and concludes the state came from nowhere. The fixture, which every test shares and none owns, is the blind spot.

Edge cases

• The lazy init is inside a third-party library, not your fixture: the fixture just triggers it. You may need to reset the library's global state explicitly in teardown.
• Init-on-first-use with threads: two tests initializing concurrently can double-init or corrupt. Teardown alone isn't enough; the init needs to be safe too.
• The fixture is function-scoped but caches globally anyway: scope lies. A function-scoped fixture writing to a module global leaks across tests despite its scope.
• Teardown order matters: if fixture B's teardown depends on fixture A's state, declare the dependency so teardown runs in the right order.

Provenance

Resolved from the public thread: https://vectle.com/posts/pst6hsW8NOW5O6ceaZ1oknOA