how to fix order-dependent test failures

TL;DR

Order dependence is pollution with a specific trigger. Run the suite in random order to surface it, fix the shared state, and keep random ordering on so it never regresses.

Error

(Not an error; a failure pattern. Symptom: the suite passes in one order and fails in another.)

Steps

1. Enable random ordering: Jest --randomize, pytest-randomly, RSpec --order random. Expected: the failure appears or moves.
2. Find the dependent pair with bisection. Expected: test A enables test B's failure.
3. Fix the shared state (see test pollution skill). Expected: the pair passes in both orders.
4. Keep random ordering permanently in CI. Expected: new order dependence is caught immediately.
5. Seed the random order for reproducibility when debugging. Expected: deterministic replays.

When to use

• Order changes results.
• After enabling random ordering for the first time.

When not to use

• Timing flakes with no order pattern.
• Suites that must run in order (rare; usually a design smell).

Tool compatibility

• Jest, pytest, RSpec random-order options.

Variant phrasings

Tests pass individually fail together

The symptom; order dependence is one cause.

Random test order failures

The diagnostic; keep it on permanently.

Why it happens

Shared state plus fixed order hides dependence. Random order exposes it.

Edge cases

• Some suites legitimately need order (migrations); isolate those files.
• Random order with a fixed seed reproduces failures deterministically.
• Parallelism is random ordering taken further; fix order first, then parallelize.

Provenance

Resolved from the public thread: https://vectle.com/posts/pst2CVof317RyWQc9nv0QZCQ