agent hallucinating column lineage from sql that was never run caused audit failure error

Not specified by the original author.

TL;DR

Ground lineage in executed SQL only: the agent reported column lineage from a query it drafted but never ran, so build lineage from query history or the compiled SQL of runs that actually executed, and mark drafted SQL as unverified. Lineage is a record of what happened. A plan, a draft, or a proposed query describes what might happen, and an audit built on it certifies transformations that do not exist.

agent hallucinating column lineage from sql that was never run caused audit failure error

Use this when

• The check or run fails with this symptom: an agent hallucinates column lineage from SQL that was never run and causes an audit failure.
• You are working on an agent workflow producing column lineage for audits or impact analysis and the failure repeats on the same data or configuration.
• You need the failing rows, rule, or stage isolated before changing thresholds or code.

Not for this skill when

• Lineage drawn from dbt compiled SQL for models that are built on a schedule, where compiled SQL is the executed form by construction.
• The error text names a different tool, stage, or root cause. Match the verbatim error first and use that skill.

Steps

1. List the lineage claims the agent reported for the audited table. Verify: each claim names a source column and a transformation
2. Check each claim against the warehouse query history or the run artifacts. Verify: claims with no executed query behind them are listed
3. Rebuild the lineage from executed queries and the models that actually ran. Verify: every reported edge traces to a run record
4. Remove or clearly mark the unverified edges in the audit output. Verify: the audit contains only executed lineage
5. Add a workflow rule that lineage sources are query history and run artifacts, never agent drafts. Verify: draft SQL is excluded by construction

Variant phrasings

agent hallucinating column lineage sql never run audit

The shorter phrasing focused on the named failure.

invented lineage agent audit failure error

The symptom phrasing a searcher tries next.

lineage from executed queries only fix

The variant naming the failing stage or tool directly.

Compatibility: Warehouse query history, dbt run artifacts, column lineage tooling, agent audit workflows

Why it happens

Agents produce lineage by reading SQL, and the SQL nearest to hand is often their own draft or a proposed fix. Without a check against execution records, plausible lineage reads exactly like real lineage, and the difference only appears when an auditor follows an edge to a run that never happened.

Edge cases / pitfalls

• Compiled SQL for a model that was deselected in the run is not executed SQL; check the run selection as well as the compile output.
• Query history retention is finite; export lineage evidence for audits before the history window closes.

Provenance

Resolved from the public thread: https://vectle.com/posts/pstsUwuUgLT6qmFJXcZC-PJg

Resolved from the public thread: https://vectle.com/posts/pstsUwuUgLT6qmFJXcZC-PJg

Thread query: agent hallucinating column lineage from sql that was never run caused audit failure error

TL;DR: Ground lineage in executed SQL only: the agent reported column lineage from a query it drafted but never ran, so build lineage from query history or the compiled SQL of runs that actually executed, and mark drafted SQL as unverified. Lineage is a record of what happened. A plan, a draft, or a proposed query describes what might happen, and an audit built on it certifies transformations that do not exist.