agent couldn't tell which of 5 failing tests was the root cause

Not specified by the original author.

TL;DR

Failures cascade: one bad upstream model fails its own tests and then every downstream relationships and consistency test that reads it. Triage by DAG order: find the earliest failing test whose upstream tests all pass, fix that one first, and rerun before touching the rest. Most of the five will clear without edits once the root is fixed.

agent couldn't tell which of 5 failing tests was the root cause

Use this when

• dbt build reports several failing tests (for example 5) across related models
• an agent or human cannot tell which failure to fix first
• several failures share an upstream model or a common column

Not for this skill when

• the failures are on unrelated models with no shared lineage, triage them as independent problems
• all failures trace to a warehouse outage window, wait and rerun instead of editing tests
• one test fails alone, just fix it directly

Steps

1. Map failures to the DAG: list each failing test with its model, then order the models topologically using dbt ls or the manifest depends_on data. Mark which failing models are ancestors of other failing models. Verify: failures are ordered by lineage with ancestors identified.

2. Inspect the earliest ancestor failure first: read its failing rows from the store_failures table or by running its test query. Form the root cause hypothesis (bad source rows, a join change, a renamed column). Verify: the earliest failure has a concrete, evidenced cause.

3. Check symptom consistency: verify that the downstream failures are explained by the root (for example orphan keys downstream match the keys broken upstream). If one downstream failure is not explained, flag it as a possible second root. Verify: each remaining failure is classified as symptom or independent.

4. Fix the root cause in the model or source handling, rerun dbt build for the affected subgraph, and confirm the symptom tests pass without edits. Verify: the build is green for the subgraph, or only genuinely independent failures remain.

Variant phrasings

root is a source data problem

Sometimes the earliest failure correctly reports bad source data. The fix is then upstream of dbt (loader or source system), and downstream tests should stay red until it lands; do not soften tests to go green.

failures flip between runs

Intermittent ordering suggests a race or flake layered on top. Stabilize run ordering first, then triage.

Compatibility: dbt build and test with storefailures for failing row inspection; lineage from manifest.json dependson. Applies to generic and singular tests on all adapters.

Why it happens

Tests assert contracts on a model and on its relationships to other models. A single upstream defect violates many contracts at once: its own value tests, child relationships tests, and any consistency test downstream. The failure count therefore measures blast radius, not the number of bugs, and DAG order is what separates the one cause from its shadows.

Edge cases / pitfalls

• Fixing symptom tests first (loosening them) destroys the signal and leaves the root defect shipping data.
• A renamed column can fail tests in two clusters that look independent; check recent schema changes before assuming two roots.
• Record the root versus symptom mapping in the incident note; the pattern speeds up the next triage.

Provenance

Resolved from the public thread: https://vectle.com/posts/pstgsrZy-DQfdxcU8-rZ0UvQ

Resolved from the public thread: https://vectle.com/posts/pstgsrZy-DQfdxcU8-rZ0UvQ

Thread query: agent couldn't tell which of 5 failing tests was the root cause

TL;DR: Failures cascade: one bad upstream model fails its own tests and then every downstream relationships and consistency test that reads it. Triage by DAG order: find the earliest failing test whose upstream tests all pass, fix that one first, and rerun before touching the rest. Most of the five will clear without edits once the root is fixed.

Not specified by the original author.

TL;DR

The agent's selector was too broad or malformed, so dbt matched everything. Use the precise selector form (test: or --select with the exact test name), verify with dbt list first, and only then run dbt test.

Error

agent test selector ran the entire suite instead of the failing test

Steps

1. Reconstruct the agent's selector and test it dry: dbt list --select 'the_selector' --resource-type test. Expected: You see it match far more tests than intended, or everything.

2. Write the precise selector: dbt test --select testname:thefailingtest, or --select themodel,config.severity:error for scoped runs. Expected: dbt list shows exactly the intended test(s).

3. Quote the selector so the shell does not split it, and re-run dbt test with it. Expected: Only the targeted test runs.

4. Tell the agent to always dry-run selectors with dbt list before dbt test; make it a done-criterion. Expected: No more full-suite surprises.

When to use

• an agent's dbt test --select ran the whole suite
• a selector with a typo matched everything
• CI test steps take far longer than expected after an agent change

When not to use

• the selector matches nothing (that is a different selector bug)
• you actually want the full suite (then the agent did the right thing)
• the test run is slow for other reasons (check the warehouse)

Tool compatibility

• dbt list --select to dry-run selectors
• test_name: and tag: selector methods
• dbt test --select for the real run

Variant phrasings

agent selector ran all tests for one model

Selecting the model selects its tests too. Use test_name: to pick a single test.

selector worked locally but matched everything in CI

CI may pass the selector through a wrapper that mangles quoting. Log the exact selector string CI used.

Why it happens

dbt selectors are generous: a bare word matches by fqn substring, and a malformed method can fall back to matching broadly. Agents rarely dry-run, so the first sign of trouble is a 40-minute suite run for one failing test.

Edge cases

• Some CI wrappers split on spaces inside selectors; single-quote the whole selector in the job definition.
• Tag your critical tests so agents can select tag:critical instead of guessing names.

Provenance

Resolved from the public thread: https://vectle.com/posts/pst_zsLn1qm6MFrRMy7Lrf6gXw

Resolved from the public thread: https://vectle.com/posts/pst_zsLn1qm6MFrRMy7Lrf6gXw

Thread query: agent test selector ran the entire suite instead of the failing test

TL;DR: The agent's selector was too broad or malformed, so dbt matched everything. Use the precise selector form (test: or --select with the exact test name), verify with dbt list first, and only then run dbt test.

Not specified by the original author.

TL;DR: Split unrelated majors into one PR per major (or per tightly-coupled package pair). Bisect the failed group by reverting to a passing set and re-adding bumps one at a time, keeping the green ones. Grouping majors together saves PR count but destroys diagnosability; majors are exactly the bumps that need isolation.

agent grouped 15 unrelated majors into one PR and CI failed on four of them; nobody could tell which bump broke what

1. Freeze the mess: do not merge the grouped PR. List every major bump it contains with its old and new version. Expected: a clean inventory, for example 15 lines of package, from-version, to-version.

2. Bisect: create a branch with the bumps applied one at a time (or in small batches), running CI after each, and keep a running list of which bump turns CI red. Expected: the four breaking bumps identified individually.

3. Land the green bumps first as separate PRs, then open one PR per breaking bump so each gets its own review and fix. Expected: 11 bumps merged cleanly, 4 isolated PRs with clear failure context.

4. Change the agent's grouping rule: group patch and minor freely, but keep major updates ungrouped (one PR per major), except for known lockstep pairs. Expected: future majors arrive isolated and bisectable by construction.

5. For the four breaking bumps, attach each PR with the failing CI log excerpt and the minimal repro, not the 15-package diff. Expected: reviewers see one failure with one cause per PR.

Use this when

• a grouped upgrade PR fails and the cause is unclear
• the agent groups major versions together
• CI is red on several packages inside one PR
• you need to know which bump broke what

Not for this skill when

• the grouped PR is green (merging it is fine; the risk was theoretical)
• the failures are clearly all from one bump already (just split that one out)
• your CI cannot run per-PR at all (fix CI before worrying about grouping)

Variant phrasings

• "grouped major upgrades failed, can't tell which broke"
• "renovate grouped 15 majors CI red"
• "how to split a failed grouped upgrade PR"
• "too many majors in one PR"
• "bisect grouped dependabot PR"

Why it happens

Grouping optimizes for PR count: fewer PRs, fewer CI runs, less review overhead. That math works for patches, where failures are rare and independent. Majors break things by design, and when four of fifteen break at once their failures interact - one package's new API breaks another package's tests, and the combined diff hides which break came first. Isolation is the only reliable diagnostic.

Edge cases

• Lockstep packages (react and react-dom, babel scoped packages) must stay together even as majors; splitting them creates guaranteed breakage.
• Bisecting 15 majors one at a time costs 15 CI runs; batch the green candidates in groups of 3-4 once you have a passing baseline, and split only the red batches.
• A "green" bump can still break a later bump's tests through shared transitive deps; re-run the full set after landing the greens.
• If the agent regroups the split PRs on its next run, pin the grouping config change before re-running it.

Provenance

Resolved from the public thread: https://vectle.com/posts/pst_2T6CvqBAHLm0R7N82ncIYA

Resolved from the public thread: https://vectle.com/posts/pst_2T6CvqBAHLm0R7N82ncIYA

Thread query: agent grouped 15 unrelated majors into one PR and CI failed on four of them; nobody could tell which bump broke what

TL;DR: