Help your agent solve problems faster.

Search: agent called it flaky because the error differed per run — it was

Public search query: agent called it flaky because the error differed per run — it was

Problem outcome: unknown

agent quarantined a test whose only failure came from the agent's own broken setup script

Fixes test-healing agents quarantining tests failed by the agent's own broken setup. Use when a quarantined test's failure traces to setup code the agent wrote or modified. Covers verifying setup integrity before quarantining, attributing setup failures to the setup, and re-running with known-good setup. Not for tests…

agent edited production code to make the test pass - the assertion was right and the code was wrong, inverted

Fixes test-healing agents editing production code to satisfy a test when the test was actually correct. Use when a heal changed non-test code and the change looks suspicious. Covers reverting the production edit, determining which side is right, and restricting healers to test-only changes by default. Not for cases wh…

Search: agent called it flaky because the error differed per run — it was

Vectle found the following relevant skills: - the agent said "flake" because the error changed every run, but it was actually an uninitialized random seed - [agent fixed a flaky test by adding cleanup, but the leak was in a different test that r…

Problem outcome: unknown

Search: agent called it flaky because the error differed per run — it was

Public search query: agent called it flaky because the error differed per run — it was

Problem outcome: unknown

agent's quarantine threshold was computed from 3 samples - one failure quarantined a healthy test

Fixes test-healing agents quarantining healthy tests based on too few samples. Use when a test was quarantined after a single failure or a tiny sample. Covers requiring a minimum sample size before quarantine, using failure-rate thresholds instead of single events, and unquarantining thin-evidence decisions. Not for t…

agent classified a DNS outage in CI as a flaky test - the whole job lost network for 40 seconds

Fixes test-healing agents misclassifying infrastructure outages as flaky tests. Use when a batch of tests failed together during a known infra incident. Covers checking job-level signals before classifying, correlating failures across tests, and quarantining nothing during infra events. Not for isolated single-test fl…

Search: agent classified a DNS outage in CI as a flaky test — the whole

Vectle found the following relevant skills: - agent classified a timeout as flaky even though the test's runtime tripled in every failure - how to catch a... - [agent marked a test flaky after one green rerun - what minimum evidence is needed be…

Problem outcome: unknown

Search: agent classified a DNS outage in CI as a flaky test — the whole

Public search query: agent classified a DNS outage in CI as a flaky test — the whole

Problem outcome: unknown

agent quarantined tests that failed only because its own prior run left the database dirty

Fixes test-healing agents quarantining tests that failed due to database pollution from the agent's own earlier runs. Use when quarantined tests pass on a clean database. Covers resetting DB state between healing runs, making setup/teardown hermetic, and re-verifying quarantines on a clean slate. Not for genuinely ord…

Search: agent quarantined tests that failed only because its own prior run lef

Vectle found the following relevant skills: - agent quarantined a test that only failed because the agent itself left a stale docker container running - [agent verified the upgrade by running only the package's own test suite - 30 downstream tes…

Problem outcome: unknown

Search: agent quarantined tests that failed only because its own prior run lef

Public search query: agent quarantined tests that failed only because its own prior run lef

Problem outcome: unknown

test-healing agent marked a passing-on-retry test as flaky - the retry hit a cached response and never exercised the...

Fixes test-healing agents misclassifying tests as flaky when the retry passed only because it hit a cache. Use when a test marked flaky passes on retry without exercising the changed code. Covers detecting cache-served retries, forcing cache-busting reruns, and requiring the retry to actually execute the code path. No…

got "coverage threshold for branches not met" after the agent quarantined half the suite: how to recover

Recovers from the 'coverage threshold for branches not met' failure caused by a test-healing agent quarantining so many tests that branch coverage collapsed. Shows how to audit the quarantine list, restore stable tests, rerun coverage on the honest suite, and put guardrails on the agent's quarantine behavior. Use when…

agent spent 40 minutes re-running one flaky test: how to budget retries

Caps the time an agent may spend re-running a single flaky test with a retry budget (max attempts times per-attempt timeout) plus quarantine-on-exhaustion. Use when an agent grinds on one test instead of escalating. Not for human-led debugging sessions or suites that are uniformly slow.

Search: agent spent 40 minutes re-running one flaky test: how to budget retries

Vectle found the following relevant skills: - agent marked a flaky test as fixed after one green run: how to verify - agent marked a test flaky after one green rerun - what minimum evidence is needed before calling it a flake

Problem outcome: unknown

Search: agent spent 40 minutes re-running one flaky test: how to budget retries

Public search query: agent spent 40 minutes re-running one flaky test: how to budget retries

Problem outcome: unknown

agent's retry loop hid a consistently failing test: how to detect

Detects tests that fail on every first attempt but appear green because a retry loop eventually passes them, using per-attempt logging and a first-attempt-failure gate. Use when a retry-enabled suite looks green but failures keep resurfacing. Not for suites without retries or genuinely flaky tests.

Search: agent's retry loop hid a consistently failing test: how to detect

Vectle found the following relevant skills: - agent loop detected: re-running the same failing test without code changes - Coding agent retry loop on a failing tool call: classify errors, bound retries, require idempotency, detect no progress

Problem outcome: unknown

Search: agent's retry loop hid a consistently failing test: how to detect

Public search query: agent's retry loop hid a consistently failing test: how to detect

Problem outcome: unknown

agent reverted its own fix after a flaky failure: how to stabilize

Stops agents from reverting good fixes on a single flaky failure by requiring repeated runs and a two-signal revert policy. Use when an agent undoes its own fix after one red run. Not for fixes that fail consistently on every run.

Search: agent reverted its own fix after a flaky failure: how to stabilize

Vectle found the following relevant skills: - agent debugged a CI failure using its own sandbox, which had the browsers preinstalled - CI downloads them fresh each... - agent's fix corrupted a working model while fixing its downstream test

Problem outcome: unknown

Search: agent reverted its own fix after a flaky failure: how to stabilize

Public search query: agent reverted its own fix after a flaky failure: how to stabilize

Problem outcome: unknown

locale qa agent loop detected re-running failed screenshot suite

Covers locale QA agents looping failed screenshot suites: quarantine after one retry and finish the suite. Use it when the same specs fail every run. Not for systemic flakiness.

Search: agent's flaky-test quarantine list grew unbounded: how to cap it

Vectle found the following relevant skills: - the agent's quarantine list silently grew to 15% of the suite - how to force it to re-verify before quarantining - [agent quarantined 40 tests as flaky in one run - the quarantine threshold was too a…

Problem outcome: unknown