agent hallucinating column names that don't exist error failed query generation

TL;DR

Ground every generated query in the live catalog: before running generated SQL, resolve each column name against the information schema for the target table and reject or correct any name that does not exist. Hallucinated columns usually are plausible names from similar tables or from an older schema version.

agent hallucinating column names that don't exist error failed query generation

Use this when

• The check or run fails with this symptom: generated queries fail because the agent invented column names that do not exist.
• You are working on an agent that writes SQL or data quality checks from natural-language requests and the failure repeats on the same data or configuration.
• You need the failing rows, rule, or stage isolated before changing thresholds or code.

Not for this skill when

• A column that exists but under a different table or schema, where the fix is qualifying the name rather than treating it as invented.
• The error text names a different tool, stage, or root cause. Match the verbatim error first and use that skill.

Steps

1. Extract every column reference from the generated query and look each one up in the information schema for its table. Verify: each reference is marked found or missing against the live catalog
2. For missing names, search the catalog for near matches by suffix, prefix, or similar tables the agent may have copied from. Verify: each missing name has a candidate real column or is confirmed absent
3. Rewrite the query with only verified columns, and where a requested field genuinely does not exist, report that instead of substituting silently. Verify: the corrected query uses catalog-verified names only
4. Run the corrected query with a small limit first and confirm it executes before the full run. Verify: the limited run returns rows with no unknown-column error

Variant phrasings

agent invented column names sql failed

The invented-column phrasing.

hallucinated columns in generated data quality checks

The generated-check phrasing.

agent query references columns that do not exist

The missing-column phrasing.

Compatibility: Agents generating SQL against warehouse schemas with Great Expectations or dbt context

Why it happens

Language models generate column names from patterns in training data and prior context, so names sound right while belonging to other tables, older versions, or no schema at all. Nothing in generation checks the catalog; verification has to be an explicit step after generation and before execution.

Edge cases / pitfalls

• A column dropped minutes ago can pass a stale cache; refresh the catalog for the target table when a lookup disagrees with a fresh error.
• Case sensitivity differs across warehouses; compare names using the warehouse's folding rules.

Provenance

Resolved from the public thread: https://vectle.com/posts/pst_Z5If9RjeWNqAEekCWaZIbA