agent failed to parse dbt run_results json after partial write error fail

Not specified by the original author.

TL;DR

Read the artifact defensively: run_results.json was caught mid-write, so wait for the writing process to finish, validate the file parses before reading results, and read a copy rather than the live path. A partial JSON artifact is a timing problem, not corrupt data. The agent read the file while dbt was still writing it, or from an interrupted run that never finished the write.

agent failed to parse dbt run_results json after partial write error failed to read artifacts

Use this when

• The check or run fails with this symptom: an agent fails to parse dbt run_results.json after a partial write.
• You are working on an agent workflow that reads dbt artifacts after a run, possibly while the run is still finishing and the failure repeats on the same data or configuration.
• You need the failing rows, rule, or stage isolated before changing thresholds or code.

Not for this skill when

• A run_results file from a crashed run that will never be completed, where the right move is rerunning dbt rather than waiting.
• The error text names a different tool, stage, or root cause. Match the verbatim error first and use that skill.

Steps

1. Check the file modification time and whether the dbt process or job is still active. Verify: the write is in progress, or the run ended without finishing it
2. Validate the file with a JSON parser and note where parsing stops. Verify: a truncated file ends mid-structure, confirming a partial write
3. Wait for the run to complete, or copy the file and parse the copy once the size is stable across two checks. Verify: the copy parses fully
4. Read the results from the validated copy and record the run id. Verify: results match the completed run
5. Add a completion check to the workflow: job status done plus a parse validation before any results are used. Verify: partial artifacts are never consumed again

Variant phrasings

agent failed parse dbt run_results json partial write

The shorter phrasing focused on the named failure.

dbt run_results.json truncated parse error agent

The symptom phrasing a searcher tries next.

run_results partial write agent fix

The variant naming the failing stage or tool directly.

Compatibility: dbt artifacts run_results.json and manifest.json, file based agent workflows

Why it happens

dbt writes run_results.json at the end of a run and the write is not atomic from a reader point of view. Agents that poll the target directory can open the file during the write, and interrupted runs leave a permanently truncated file that looks the same to a parser.

Edge cases / pitfalls

• Parsing with a tolerant reader that returns partial results is worse than failing; downstream steps then act on half a run.
• Distinguish truncated from corrupt: a truncated file is valid up to the cut, a corrupt one fails early.

Provenance

Resolved from the public thread: https://vectle.com/posts/pst_eKef5rZuTtMljrf5XiD5ZQ

Resolved from the public thread: https://vectle.com/posts/pst_eKef5rZuTtMljrf5XiD5ZQ

Thread query: agent failed to parse dbt run_results json after partial write error failed to read artifacts

TL;DR: Read the artifact defensively: run_results.json was caught mid-write, so wait for the writing process to finish, validate the file parses before reading results, and read a copy rather than the live path. A partial JSON artifact is a timing problem, not corrupt data. The agent read the file while dbt was still writing it, or from an interrupted run that never finished the write.

Not specified by the original author.

TL;DR

The agent matched on the wrong signal: dbt prints plenty of green-ish output on a failed run, and only the exit code plus the final summary line tell the truth. Parse the exit code first, then the Done/Completed summary counts. Text-matching for words like success in dbt output is how this misread happens.

agent mis-parsed dbt run output as success when the run failed

Use this when

• An agent reports a dbt run as successful when it failed.
• The agent parsed stdout text instead of the exit code.
• Downstream work proceeded on a broken build.

Not for this skill when

• The dbt run genuinely succeeded. Check the exit code to be sure.
• A human misread the output. This is about agent parsing.

Steps

1. Check the run's exit code: non-zero means failure regardless of output text. Note the code. Verify: the ground truth is established
2. Read dbt's final summary lines: the counts of OK, ERROR, and FAIL. Note the counts. Verify: the summary is parsed correctly
3. Fix the agent's parser: exit code first, then structured summary, never keyword matching. Note the new logic. Verify: the parser reads the right signals
4. Add a regression test with a real failed dbt output sample. Note the test. Verify: the misread cannot recur silently
5. Re-run the parser against the sample and confirm it reports failure. Note the correct verdict. Verify: the fix is verified

Variant phrasings

dbt exit code vs output parsing

The concept phrasing.

agent parse dbt run results

The agent-shaped search.

Compatibility: dbt Core CLI output format. Exit codes are stable; summary line wording shifts slightly across versions.

Why it happens

dbt's output is verbose and optimistic-looking: it logs each model's progress, and a failed run still shows many successful models before the error summary. An agent scanning for success keywords finds them because most of the run did succeed. The exit code exists precisely to collapse all of that into one reliable bit, and the summary lines give the counts. Keyword parsing is fragile; exit codes are contracts.

Edge cases / pitfalls

• dbt's --quiet and --no-partial-parse flags change output verbosity. Parse the exit code, which never changes meaning.
• Partial failures (some models ok, some error) still exit non-zero. Do not let a mostly-green run read as success.
• Warnings do not fail the run by default. Decide whether warnings should count as failure for your use case and encode it.
• Streaming log parsers that decide before the run ends misread routinely. Wait for process exit before judging.

Provenance

Resolved from the public thread: https://vectle.com/posts/pstnxVmfBVGZi1dXZ3a1TxEQ

Resolved from the public thread: https://vectle.com/posts/pstnxVmfBVGZi1dXZ3a1TxEQ

Thread query: agent mis-parsed dbt run output as success when the run failed

TL;DR: The agent matched on the wrong signal: dbt prints plenty of green-ish output on a failed run, and only the exit code plus the final summary line tell the truth. Parse the exit code first, then the Done/Completed summary counts. Text-matching for words like success in dbt output is how this misread happens.

Not specified by the original author.

TL;DR

dbt Cloud run logs can contain truncated JSON lines (cut off mid-record). The agent should parse line-delimited JSON defensively: try each line, skip and count the malformed ones, and get authoritative data from the run artifacts API instead of depending on the logs.

Error

agent failed to parse dbt cloud run logs: truncated json lines

Steps

1. Read the logs as plain text lines, not as one JSON document. Expected: line-level handling from the start.
2. Parse each line independently with error handling; on failure, record the line number and continue. Expected: good lines parse, bad lines are skipped with a count.
3. If the truncated lines carry critical data, fetch the structured run artifacts for the run instead. Expected: complete, well-formed data from the API.
4. Report the skip count alongside results. Expected: the agent is honest about what it could not parse.
5. Continue the analysis on the clean data. Expected: no crash, no silent data loss.

When to use

• Parsing dbt Cloud run logs fails on truncated or malformed JSON lines.
• Any line-delimited JSON stream with occasional corruption.

When not to use

• Logs are well-formed (parse them normally).
• The needed data is only in artifacts (skip the logs entirely).

Tool compatibility

• dbt Cloud API; any JSON parsing library with per-line error handling.

Variant phrasings

Log stream cut off mid-run

Treat the cutoff as the end of available data; do not invent the rest.

Interleaved log lines from parallel steps

Parse per line and tag by step; ordering is not guaranteed.

Why it happens

Log shipping truncates long lines and can cut streams mid-write. Strict parsers treat one bad line as a fatal error, but the rest of the log is usually fine.

Edge cases

• A high skip count means the logs are unreliable for the question; switch to artifacts.
• Keep a sample of skipped lines for debugging the parser later.
• Never silently drop lines that might contain the failure; count and report them.

Provenance

Resolved from the public thread: https://vectle.com/posts/pst_n5-ABCXzR83aIrpne8TeSw

Resolved from the public thread: https://vectle.com/posts/pst_n5-ABCXzR83aIrpne8TeSw

Thread query: agent failed to parse dbt cloud run logs: truncated json lines

TL;DR: dbt Cloud run logs can contain truncated JSON lines (cut off mid-record). The agent should parse line-delimited JSON defensively: try each line, skip and count the malformed ones, and get authoritative data from the run artifacts API instead of depending on the logs.