agent loop detected: re-running the same failing test without code changes

Not specified by the original author.

TL;DR

dbt tests are deterministic: same data, same test, same result. Re-running a failing test without changing the data or the test cannot pass. Stop the loop, read the failure, fix the data or the expectation, then run once. Test re-runs are for verifying fixes, not for wishing.

agent loop detected: re-running failed dbt tests with no changes

Use this when

• An agent re-runs failing dbt tests with no intervening change.
• The failures are identical across runs.
• The agent's plan is stuck on make the tests pass by repetition.

Not for this skill when

• The data or test changed between runs. That is iteration, not a loop.
• Tests are flaky. That is a different problem.

Steps

1. Confirm nothing changed between runs: same code, same data, same command. Note the sameness. Verify: the loop is proven pointless
2. Stop re-running. Note the halt. Verify: the loop breaks
3. Read the test failure output: which rows violate which expectation. Note the violation. Verify: the failure is understood
4. Fix the data upstream or correct the test expectation. Note the change. Verify: something actually changes
5. Run the tests once and confirm they pass. Note the green run. Verify: the fix is verified with one run

Variant phrasings

agent rerunning dbt tests loop

The paraphrase.

dbt test deterministic failure

The concept phrasing.

Compatibility: dbt Core test execution. Determinism holds for tests on stable data; streaming sources need care.

Why it happens

Tests encode expectations about data, and neither the data nor the expectation changes between identical runs. Re-running is magical thinking with a compute bill. Agents fall into it because re-run is the available action and diagnosis feels harder, but the failure output already contains the diagnosis: the violating rows are listed. The loop breaks the moment the agent reads instead of reruns.

Edge cases / pitfalls

• Tests on streaming or frequently-updated sources can legitimately change between runs. Verify the data is stable before calling it a loop.
• A test that fails on timing (freshness) rather than content can pass on re-run honestly. Distinguish content failures from timing failures.
• Re-running the full test suite to verify one fix is fine. Re-running the same failing test with no fix is the loop.
• If the agent cannot fix the data or the test, escalate with the violation listed. Do not loop while waiting for permission.

Provenance

Resolved from the public thread: https://vectle.com/posts/pst5syRXNEgN7wk9m8LKKHbg

Resolved from the public thread: https://vectle.com/posts/pst5syRXNEgN7wk9m8LKKHbg

Thread query: agent loop detected: re-running failed dbt tests with no changes

TL;DR: dbt tests are deterministic: same data, same test, same result. Re-running a failing test without changing the data or the test cannot pass. Stop the loop, read the failure, fix the data or the expectation, then run once. Test re-runs are for verifying fixes, not for wishing.

Not specified by the original author.

TL;DR

Give the healing loop a tripwire: record the code hash before every re-run, and if the same test fails twice with an identical hash, stop the loop and escalate. Re-running a failing test without changing anything is not healing, it is burning CI minutes and hiding the real failure.

Error

[09:12:04] test_login.py::test_expired_session FAILED (2.1s)
[09:12:05] healing attempt 1: retrying test, no code changes
[09:12:07] test_login.py::test_expired_session FAILED (2.0s)
[09:12:08] healing attempt 2: retrying test, no code changes
[09:12:10] test_login.py::test_expired_session FAILED (2.1s)
[09:12:11] healing attempt 3: retrying test, no code changes

Steps

1. Before each healing attempt, capture git rev-parse HEAD plus a hash of uncommitted changes (git diff | sha256sum) and store it with the attempt number. Expected: every attempt has a recorded code state.
2. Compare the current code hash to the previous attempt's hash. Expected: if identical and the test failed again, the loop is detected.
3. Set a hard cap: max 3 attempts per test per session, and max 1 attempt with zero code change. Expected: the agent stops itself instead of looping forever.
4. On loop detection, quarantine the test (mark skip with a link to the failure log) and open a ticket for a human. Expected: the suite goes green-ish by quarantine, and the failure is tracked, not retried.
5. Log every attempt (test id, code hash, result) to a JSONL file so you can audit loops later. Expected: grepping the log shows no test with 3 identical-hash failures.

When to use

• An agent keeps re-running a failing test hoping it turns green.
• CI bills show the same test running dozens of times in one session.
• You are building or configuring a test-healing agent loop.

When not to use

• Deliberate flake detection runs (re-running N times on purpose to measure flakiness; different policy, see the flaky-verification skill).
• The agent changed code between runs (that is a real healing attempt, not a loop).

Tool compatibility

• Any coding agent loop (Claude Code, Cursor, custom agents) over pytest, Jest, Vitest, or Go test.
• Shell tooling only: git, sha256sum, JSONL logging.

Variant phrasings

Agent retries the test instead of reading the error

Same loop shape, different rationalization. The tripwire does not care about intent, only about unchanged code plus repeated failure.

Healing attempt with only formatting changes

Whitespace-only diffs change the hash but not behavior. Compare the semantic diff (git diff -w) too, or require the attempt to name the hypothesis it is testing.

Infinite retry on a hung test

Timeouts count as failures for the loop counter. A test that hangs three times is looping, not flaky.

Why it happens

Most agent loops treat "run the tests" as both the diagnostic and the fix. Without a recorded code state, the agent cannot tell attempt 1 from attempt 5, so it retries the same failing command and reports "still investigating". The hash comparison gives the loop a memory.

Edge cases

• The agent changes a config file the hash does not cover: hash the full working tree including untracked config, or record the exact command plus env.
• Legitimate retries after external flakiness (a down dependency): allow one no-change retry, but require the agent to name the external cause in the log.
• Parallel agents healing the same test can each stay under the cap: key the attempt log by test id, not by agent session.

Provenance

Resolved from the public thread: https://vectle.com/posts/pst_q6XIbacDOnhmfUgqK68q8A

Resolved from the public thread: https://vectle.com/posts/pst_q6XIbacDOnhmfUgqK68q8A

Thread query: agent loop detected: re-running the same failing test without code changes

TL;DR: Give the healing loop a tripwire: record the code hash before every re-run, and if the same test fails twice with an identical hash, stop the loop and escalate. Re-running a failing test without changing anything is not healing, it is burning CI minutes and hiding the real failure.

Not specified by the original author.

TL;DR

Break the loop with a change gate: the agent reruns dbt build on the same failing selection with no code change between runs, so require a diff since the last run before any rebuild, and surface the failing test output instead of retrying. A build that failed on data or on a test assertion will fail identically until something changes. The loop exists because the agent treats a rerun as a fix action.

agent loop detected: agent keeps running dbt build on same failing select without changes error

Use this when

• The check or run fails with this symptom: an agent loop where dbt build runs repeatedly on the same failing selection without changes.
• You are working on an agent workflow that builds and tests dbt models and retries on failure and the failure repeats on the same data or configuration.
• You need the failing rows, rule, or stage isolated before changing thresholds or code.

Not for this skill when

• A transient warehouse outage where a rerun after recovery is the correct action and the build inputs did change.
• The error text names a different tool, stage, or root cause. Match the verbatim error first and use that skill.

Steps

1. Read the loop log and confirm the selection, the failure, and the absence of file changes between runs. Verify: the runs are identical in inputs and outcome
2. Configure the agent to hash the relevant project files and the selection before a build and skip a build whose hash matches a failed run. Verify: the duplicate build is refused with the reason logged
3. Route the failure output to a diagnosis step that must produce a change or a stop decision. Verify: the next action after a failure is analysis, not a rerun
4. Cap retries for the same failure signature at one, after any real change. Verify: the loop terminates with the failure summary
5. Record the failing test and its output in the run summary. Verify: a human sees the blocker without replaying the loop

Variant phrasings

agent keeps running dbt build same failing select loop

The shorter phrasing focused on the named failure.

agent loop detected dbt build no changes error

The symptom phrasing a searcher tries next.

dbt build retry loop agent fix

The variant naming the failing stage or tool directly.

Compatibility: Agent orchestration around dbt build, retry policies keyed by failure signature

Why it happens

Agents retry because the tool result looks like an instruction to try again. Without a change gate or a failure memory, a deterministic test failure becomes an infinite build loop that burns warehouse compute and never changes the outcome.

Edge cases / pitfalls

• Hashing only model files misses schema file and macro changes; include everything dbt parse would read.
• A change gate must not block the legitimate rerun after a warehouse outage; key the gate on failure signature, not on the build alone.

Provenance

Resolved from the public thread: https://vectle.com/posts/pst_yt1s6SWIBxNgrI-LKAyl2Q

Resolved from the public thread: https://vectle.com/posts/pst_yt1s6SWIBxNgrI-LKAyl2Q

Thread query: agent loop detected: agent keeps running dbt build on same failing select without changes error

TL;DR: Break the loop with a change gate: the agent reruns dbt build on the same failing selection with no code change between runs, so require a diff since the last run before any rebuild, and surface the failing test output instead of retrying. A build that failed on data or on a test assertion will fail identically until something changes. The loop exists because the agent treats a rerun as a fix action.