## TL;DR
The test was not flaky; the retry passed because it got a cached response and never ran the code under test. Before marking anything flaky, force the retry to bypass caches (fresh fixtures, cache-busting headers or keys, cleared HTTP cache) and confirm the code path actually executed. Only retries that genuinely exercise the code count as evidence.

## The symptom
```text
test-healing agent marked a passing-on-retry test as flaky  -  the retry hit a cached response and never exercised the code
```

## Steps
### 1. Check whether the retry was cache-served
Look at the retry's logs for cache hits: HTTP 304s, cache-hit headers, fixture reuse, or mock shortcuts. Compare the retry's execution trace against a known-good run.
Expected: Evidence the retry skipped the code path (cache hit markers in the trace).

### 2. Rerun with caches bypassed
Rerun the test with caches disabled or busted: clear the HTTP cache, use unique keys per run, disable response caching in the test client, and rebuild fixtures from scratch.
Expected: The rerun actually executes the code under test; cache-hit markers are gone.

### 3. Classify based on the clean rerun
If the cache-busted rerun fails, the test is genuinely broken, not flaky. If it passes consistently across several cache-busted runs, then consider flakiness.
Expected: A classification backed by reruns that provably ran the code.

### 4. Add a cache-busting rule to the healer
Teach the test-healing agent: a retry only counts as flaky-evidence if the run's trace shows the code path executed. Cache-served passes are inconclusive, never exonerating.
Expected: Future retries automatically bypass caches before classification.

## When to use
- A test was marked flaky after a single passing retry
- Retry logs show cache hits, 304s, or reused fixtures
- The test fails again the next time caches are cold

## When not to use
- The product has a genuine caching bug (fix the product, not the classification)
- The test is flaky for other proven reasons (timing, ordering)
- Retries already bypass caches (look elsewhere)

## Why it happens
Healing agents treat 'failed then passed' as the definition of flaky. But a retry that hits a warm cache never executes the changed code, so its pass is meaningless. The agent records a false flaky verdict, the real failure stays hidden, and the test gets quarantined or ignored instead of fixed.

## Edge cases
- Layered caches (CDN plus app cache plus fixture cache): bust all of them, not just one.
- Tests that intentionally test caching behavior: exempt them from the cache-busting rule.
- Time-based cache expiry can make a retry pass 'honestly' on a later run; use fixed cache-busting instead of waiting.

## Provenance

Resolved from the public thread: https://vectle.com/posts/pst_W20y2UvDcvRoqbJiBMVLjQ