agent mis-parsed dbt run output as success when the run failed

TL;DR

The agent matched on the wrong signal: dbt prints plenty of green-ish output on a failed run, and only the exit code plus the final summary line tell the truth. Parse the exit code first, then the Done/Completed summary counts. Text-matching for words like success in dbt output is how this misread happens.

agent mis-parsed dbt run output as success when the run failed

Use this when

• An agent reports a dbt run as successful when it failed.
• The agent parsed stdout text instead of the exit code.
• Downstream work proceeded on a broken build.

Not for this skill when

• The dbt run genuinely succeeded. Check the exit code to be sure.
• A human misread the output. This is about agent parsing.

Steps

1. Check the run's exit code: non-zero means failure regardless of output text. Note the code. Verify: the ground truth is established
2. Read dbt's final summary lines: the counts of OK, ERROR, and FAIL. Note the counts. Verify: the summary is parsed correctly
3. Fix the agent's parser: exit code first, then structured summary, never keyword matching. Note the new logic. Verify: the parser reads the right signals
4. Add a regression test with a real failed dbt output sample. Note the test. Verify: the misread cannot recur silently
5. Re-run the parser against the sample and confirm it reports failure. Note the correct verdict. Verify: the fix is verified

Variant phrasings

dbt exit code vs output parsing

The concept phrasing.

agent parse dbt run results

The agent-shaped search.

Compatibility: dbt Core CLI output format. Exit codes are stable; summary line wording shifts slightly across versions.

Why it happens

dbt's output is verbose and optimistic-looking: it logs each model's progress, and a failed run still shows many successful models before the error summary. An agent scanning for success keywords finds them because most of the run did succeed. The exit code exists precisely to collapse all of that into one reliable bit, and the summary lines give the counts. Keyword parsing is fragile; exit codes are contracts.

Edge cases / pitfalls

• dbt's --quiet and --no-partial-parse flags change output verbosity. Parse the exit code, which never changes meaning.
• Partial failures (some models ok, some error) still exit non-zero. Do not let a mostly-green run read as success.
• Warnings do not fail the run by default. Decide whether warnings should count as failure for your use case and encode it.
• Streaming log parsers that decide before the run ends misread routinely. Wait for process exit before judging.

Provenance

Resolved from the public thread: https://vectle.com/posts/pstnxVmfBVGZi1dXZ3a1TxEQ