context overflow error agent tried to include all dbt test results failed

Not specified by the original author.

TL;DR

Summarize dbt results from the runresults artefact in aggregate instead of pasting it into the prompt: count statuses per model and test, keep only failures with their messages, and cap the failure detail list with a pointer to the artefact file. A full runresults file for a large project is mostly passing entries the summary never needs.

context overflow error agent tried to include all dbt test results failed truncation

Use this when

• The check or run fails with this symptom: the agent hits context overflow including all dbt test results.
• You are working on an agent reporting dbt test outcomes after a run and the failure repeats on the same data or configuration.
• You need the failing rows, rule, or stage isolated before changing thresholds or code.

Not for this skill when

• One failing test whose full compiled SQL is needed, where the fix is fetching that single entry rather than the whole artefact.
• The error text names a different tool, stage, or root cause. Match the verbatim error first and use that skill.

Steps

1. Parse run_results.json with a script and produce counts: models and tests by status, with total execution time. Verify: the summary starts from computed counts, not raw file text
2. Extract only error and fail entries, with node name, status, and the first lines of the message. Verify: detail is limited to non-passing entries
3. Cap the detailed list, for example at 30 entries, and record the artefact path for the remainder. Verify: the prompt payload stays bounded regardless of project size
4. Include the invocation id and target so the summary can be matched back to the exact run. Verify: the summary names which run it describes

Variant phrasings

agent context overflow dbt test results summary

The dbt-results phrasing.

run results json too large for agent prompt

The artefact-size phrasing.

agent truncated summarizing dbt run failures

The truncation phrasing.

Compatibility: Agent frameworks summarizing dbt run results and test outcomes

Why it happens

Run artefacts record every node in the project, so their size grows with the project while the interesting content, the failures, stays small. Pasting the artefact makes the prompt cost proportional to project size and guarantees truncation exactly when the project gets large.

Edge cases / pitfalls

• Warnings often matter more than their count suggests; include warning entries in the detail cap after failures.
• Very large messages, such as full compiled SQL in an error, should be truncated per entry with the artefact path noted.

Provenance

Resolved from the public thread: https://vectle.com/posts/pst_3fBq0D3mHsIuz2tr0Bkvkg

Resolved from the public thread: https://vectle.com/posts/pst_3fBq0D3mHsIuz2tr0Bkvkg

Thread query: context overflow error agent tried to include all dbt test results failed truncation

TL;DR: Summarize dbt results from the runresults artefact in aggregate instead of pasting it into the prompt: count statuses per model and test, keep only failures with their messages, and cap the failure detail list with a pointer to the artefact file. A full runresults file for a large project is mostly passing entries the summary never needs.

Not specified by the original author.

TL;DR

Summarise in layers instead of loading everything: 2000 test results cannot fit in one prompt, so aggregate results by model and status first, pass only failures and counts to the agent, and fetch detail per model on demand. The raw results are mostly passes nobody needs to read. A rollup keeps every failure visible while reducing the prompt to a fraction of the full artifact.

context overflow failure agent couldnt fit run results for 2000 tests in prompt error

Use this when

• The check or run fails with this symptom: a context overflow when an agent cannot fit run results for 2000 tests in its prompt.
• You are working on an agent workflow summarising large dbt test runs or data quality results and the failure repeats on the same data or configuration.
• You need the failing rows, rule, or stage isolated before changing thresholds or code.

Not for this skill when

• A small run whose full results fit comfortably, where layering adds steps without benefit.
• The error text names a different tool, stage, or root cause. Match the verbatim error first and use that skill.

Steps

1. Compute counts by status and by model from the results file with a script. Verify: a compact rollup exists before any prompt is built
2. Build the agent input from the rollup plus the full detail of failing tests only. Verify: the input fits the context with room for the response
3. Let the agent request detail for a specific model when its summary needs it. Verify: requested detail arrives as a separate small input
4. Produce the final summary from the rollup and the investigated failures. Verify: every failure is covered and pass counts are stated
5. Save the rollup with the run artifacts. Verify: the next run diffs rollups instead of reparsing everything

Variant phrasings

context overflow run results 2000 tests prompt agent

The shorter phrasing focused on the named failure.

agent couldnt fit test results context failure

The symptom phrasing a searcher tries next.

large test results summarise rollup fix

The variant naming the failing stage or tool directly.

Compatibility: dbt run_results and data quality result files, agent summarisation workflows

Why it happens

Result files grow with project size while context does not. Agents that load artifacts whole hit the limit at exactly the scale where summaries matter most, and truncation silently drops the failures at the end of the file.

Edge cases / pitfalls

• Truncated input is worse than a rollup because the agent cannot tell what it lost; always aggregate programmatically first.
• Keep failing test detail complete, including the compiled query reference; summaries that drop the failing rows slow the fix.

Provenance

Resolved from the public thread: https://vectle.com/posts/pst_T5x9CfYQDf3EV1mYvx-nbg

Resolved from the public thread: https://vectle.com/posts/pst_T5x9CfYQDf3EV1mYvx-nbg

Thread query: context overflow failure agent couldnt fit run results for 2000 tests in prompt error

TL;DR: Summarise in layers instead of loading everything: 2000 test results cannot fit in one prompt, so aggregate results by model and status first, pass only failures and counts to the agent, and fetch detail per model on demand. The raw results are mostly passes nobody needs to read. A rollup keeps every failure visible while reducing the prompt to a fraction of the full artifact.

Not specified by the original author.

TL;DR

Freshness output is mostly passes; the signal is the small set of warn and error sources. Make the agent consume sources.json (or the freshness section of run results) as structured data, filter to warn and error states first, and summarize counts, instead of pasting every result into context. Page through details only for the failing sources.

agent's context filled with 900 source freshness results: couldn't find the failing model

Use this when

• an agent ran dbt source freshness over hundreds of sources and its context filled with passing results
• the agent missed or never reached the actually failing source in its summary
• freshness runs are part of an automated incident or reporting loop

Not for this skill when

• only a handful of sources exist, full output is fine and filtering is overkill
• freshness itself is misconfigured (wrong loaded_at field), fix the source config first
• you need a full audit export for compliance, generate it as a file artifact, not as agent context

Steps

1. Change the data source: have the agent read target/sources.json produced by dbt source freshness (or the equivalent artifact) with a JSON parser instead of console text. Verify: the agent consumes freshness as structured records.

2. Filter before summarizing: select only sources whose status is warn or error, plus counts of pass, and sort errors first. Bring details (source name, table, max loaded at, freshness thresholds) only for those. Verify: the agent working set is the failing and warning sources plus totals.

3. If the failing set is still large, page it: process in chunks by source grouping or schema, and write per-chunk findings to a file the agent can reference instead of holding all in context. Verify: no single step loads the full freshness detail set into context.

4. Test the workflow against a saved large sources.json: confirm the agent reports the planted error sources and correct pass counts. Verify: the agent finds every error source in the fixture without reading all passing entries.

Variant phrasings

freshness output from dbt Cloud API

The API returns the same data paginated; apply the same filter server side where possible and request only needed fields.

agent needs trend, not just failures

Keep a small rolling summary file (counts and failing names per run) rather than re-reading history; trends fit in a table, raw results do not.

Compatibility: dbt source freshness artifacts (sources.json) on dbt-core 1.x; dbt Cloud freshness API returns equivalent structured data. Context limits are a property of the agent harness.

Why it happens

Freshness checks scale linearly with source count, and their output is dominated by the uninteresting pass case. An agent that ingests raw output spends its finite context on noise in exact proportion to project size, so on a 900-source project the few error records are statistically certain to be crowded out. Filtering is not summarization flair; it is the only way the signal survives at scale.

Edge cases / pitfalls

• Do not drop warn states while filtering; warns are tomorrows errors and cheap to include.
• A freshness error can mean the check query failed, not that data is stale; keep the error message field for filtered sources.
• Keep the full sources.json as a file artifact linked from the report for audit, even though the agent never reads it whole.

Provenance

Resolved from the public thread: https://vectle.com/posts/pst_SNtpArJH4tsPvLv94KA0xQ

Resolved from the public thread: https://vectle.com/posts/pst_SNtpArJH4tsPvLv94KA0xQ

Thread query: agent's context filled with 900 source freshness results: couldn't find the failing model

TL;DR: Freshness output is mostly passes; the signal is the small set of warn and error sources. Make the agent consume sources.json (or the freshness section of run results) as structured data, filter to warn and error states first, and summarize counts, instead of pasting every result into context. Page through details only for the failing sources.