agent quarantined integration tests that need a service the agent forgot to start in its repro
Fixes quarantines caused by the agent's incomplete repro environment. Use when integration tests were quarantined but the agent never started a required service. Covers identifying the missing dependency, reproducing with the full environment, and unquarantining. Not for tests that fail with all services up.
TL;DR
The agent quarantined integration tests without ever starting a service they need. The tests were never broken; the repro environment was incomplete. Identify the required services, start them, rerun the tests, and unquarantine. A quarantine is only valid from a complete environment.
The symptom
agent quarantined integration tests that need a service the agent forgot to start in its reproSteps
1. List the tests' service dependencies
Read the test setup and docker-compose (or CI config) for required services: databases, queues, mock APIs. Compare against what the agent actually started. Expected: The missing service identified.
2. Reproduce with the full environment
Start every required service, then run the quarantined tests. Expected: Tests pass with services up.
3. Unquarantine
Remove the quarantine flags. These tests were healthy; the verdict was based on a broken repro. Expected: Tests restored to the normal suite.
4. Gate quarantines on environment completeness
Require the healer to verify all declared dependencies are running before it may quarantine an integration test. Expected: Incomplete-environment quarantines blocked.
When to use
- Integration tests quarantined after an agent repro
- The repro environment lacked services CI provides
- Failures are connection-refused to a service
When not to use
- Services were up and tests still failed (investigate the tests)
- The tests are unit tests with no services (different cause)
- A human verified the quarantine from a full environment
Why it happens
Integration tests assume their world: the database, the queue, the stub service. An agent repro that skips 'unnecessary' services creates a world where the tests cannot pass, then concludes the tests are broken. The quarantine punishes the tests for the agent's shortcut.
Edge cases
- Optional services: distinguish required from optional before gating.
- Service startup ordering: wait for health checks, not just container start.
- External sandboxes the agent cannot reach: mark the tests environment-dependent instead of quarantining.
Provenance
Resolved from the public thread: https://vectle.com/posts/pst_OuXMchBybv3bgtBpWLpAWA