agent hallucinating dbt test names in schema yml caused compilation failure error

TL;DR

Validate every invented name against the project: the agent wrote test names into schema.yml that do not exist as generic tests, so check each test name against the installed generic tests and the project test definitions before compiling. dbt fails compilation on an unknown generic test name. An agent generating schema files from memory will eventually invent a plausible name that no package provides.

agent hallucinating dbt test names in schema yml caused compilation failure error

Use this when

• The check or run fails with this symptom: an agent hallucinates dbt test names in schema.yml and causes a compilation failure.
• You are working on an agent workflow that authors or edits dbt schema files with generic tests and the failure repeats on the same data or configuration.
• You need the failing rows, rule, or stage isolated before changing thresholds or code.

Not for this skill when

• A test name that exists but is provided by a package that is not installed, which needs the package added rather than the name changed.
• The error text names a different tool, stage, or root cause. Match the verbatim error first and use that skill.

Steps

1. Read the compilation error for the exact test name dbt rejected. Verify: the unknown name is identified
2. List the generic tests available from dbt core and the installed packages, plus project-defined generic tests. Verify: the valid name set is known
3. Replace the invented name with the closest real test, or define the intended test as a project generic test. Verify: schema.yml references only existing tests
4. Run dbt parse. Verify: compilation succeeds with the corrected names
5. Add a validation step that checks generated schema files against the available test list before committing them. Verify: a new invented name is caught at generation time

Variant phrasings

agent hallucinating dbt test names schema yml compilation

The shorter phrasing focused on the named failure.

dbt schema yml unknown generic test agent error

The symptom phrasing a searcher tries next.

invented dbt test name compilation failure fix

The variant naming the failing stage or tool directly.

Compatibility: dbt Core generic tests, dbt_utils and project-defined generic tests, agent authored schema files

Why it happens

Generic test names look like natural language, so a generating agent produces convincing names for tests that were never implemented. dbt resolves names strictly against installed definitions, and one invented name fails the whole project compilation.

Edge cases / pitfalls

• Similar names with different argument shapes, such as relationships versus a custom foreign key test, are not interchangeable; check arguments as well as names.
• Project-defined generic tests live in the project macros or tests directories; include them in the valid set or the validator flags real tests.

Provenance

Resolved from the public thread: https://vectle.com/posts/pst_MIzzQQO2lAW0TEN3q8663A