artifact lineage breaks when model file is overwritten in place error

Not specified by the original author.

TL;DR

Never overwrite a model artifact in place: overwriting the file breaks the lineage link between the run and the artifact it produced, so version every artifact write and record the new version as a new artifact of the run or its successor. Trackers and registries reference artifacts by location and digest. Replace the file at that location and the reference now points at different bytes than the run produced, with no record of the change.

artifact lineage breaks when model file is overwritten in place error

Use this when

• The check or run fails with this symptom: artifact lineage breaks when a model file is overwritten in place.
• You are working on an experiment workflow storing model files in shared locations referenced by runs or a registry and the failure repeats on the same data or configuration.
• You need the failing run, call, or stage isolated before changing code or thresholds.

Not for this skill when

• A deliberate hotfix to an artifact, which should still be a new version with a note, never a silent overwrite.
• The error text names a different tool, stage, or root cause. Match the verbatim error first and use that skill.

Steps

1. Compare the stored digest of the artifact with the digest recorded at run time. Verify: the digests differ, proving an overwrite
2. Identify what wrote to the location after the run, from file history or job logs. Verify: the overwriting job is named
3. Restore the original artifact from the run outputs or a backup, under its original location. Verify: the digest matches the run record again
4. Move the newer file to a versioned location and log it as a new artifact linked to its own run. Verify: both versions exist with their own lineage
5. Make artifact locations write-once in the pipeline, with versioned names per run. Verify: an overwrite attempt fails loudly

Variant phrasings

artifact lineage breaks model file overwritten in place

The shorter phrasing focused on the named failure.

model artifact overwritten lineage error

The symptom phrasing a searcher tries next.

versioned artifact storage lineage fix

The variant naming the failing stage or tool directly.

Compatibility: MLflow artifacts, W&B artifacts, and model registry lineage

Why it happens

Shared output paths are convenient and corrosive: the next run, a cleanup job, or a manual copy reuses the obvious filename and replaces the bytes a run record points to. Lineage systems track the reference, not the content, so the break is invisible until someone downloads the artifact and gets a different model.

Edge cases / pitfalls

• Content addressed or versioned storage prevents the class entirely; path discipline only reduces it.
• Checksums recorded at log time are the only reliable detector; record them for every promoted artifact.

Provenance

Resolved from the public thread: https://vectle.com/posts/pstMffwMX6myiIlJMu6uYAfg

Resolved from the public thread: https://vectle.com/posts/pstMffwMX6myiIlJMu6uYAfg

Thread query: artifact lineage breaks when model file is overwritten in place error

TL;DR: Never overwrite a model artifact in place: overwriting the file breaks the lineage link between the run and the artifact it produced, so version every artifact write and record the new version as a new artifact of the run or its successor. Trackers and registries reference artifacts by location and digest. Replace the file at that location and the reference now points at different bytes than the run produced, with no record of the change.

Not specified by the original author.

TL;DR

Renaming the .sql file renames the model, but every ref('old_name') downstream still points at the old name and compilation fails with a depends-on-missing-node error. Either rename back, or update all refs to the new name and recompile the whole downstream graph. Use dbt rename-aware search (grep for the old name across all file types) so YAML, tests, and exposures are caught too.

agent renamed the model file but missed a downstream ref()

Use this when

• after a model file rename, dbt compile fails saying a model depends on a node named with the old name that was not found
• an agent renamed a file via the filesystem instead of a refactor that updates refs
• the old name still appears in ref calls, YAML docs, or exposures

Not for this skill when

• the model was deleted on purpose and downstream models should be deleted or repointed as a design change
• the missing node is a source, not a model, fix the source definition
• two models now share the new name, resolve the duplicate first

Steps

1. Inventory the old name: grep the entire project for the old model name across .sql, .yml, and .md files, including ref calls, model lists, and exposure depends_on entries. Verify: every textual reference to the old name is listed.

2. Decide direction: if the rename was intentional and approved, update each reference to the new name; if it was an agent accident, rename the file back to the original name and discard reference edits. Verify: one consistent name is chosen for the model everywhere.

3. Apply the changes, then run dbt parse and dbt ls to confirm the model exists under the chosen name with its full downstream graph intact. Verify: parse succeeds and the model shows the expected children in dbt ls output.

4. Compile the downstream chain with dbt compile --select using the model name with a trailing plus to include children, and run its tests. Verify: the downstream graph compiles and tests pass.

Variant phrasings

rename also changed the folder

Moving folders can drop folder-level config (materialization, tags, schema) from dbt_project.yml. Reapply config for the new path in the same change.

versioned model rename

Renaming a versioned model touches its version YAML entries as well; update the model name and keep versions mapping consistent.

Compatibility: dbt-core model naming by filename and ref resolution across SQL and YAML. Applies to core projects and dbt Cloud IDE renames alike.

Why it happens

dbt binds model identity to the filename at parse time, while references bind by the name string at compile time. A filesystem rename updates the first binding and none of the second, so the project contains pointers to a node that no longer exists. The compiler error is precise; the miss is purely that rename and reference updates were not done as one atomic change.

Edge cases / pitfalls

• git mv preserves history for the renamed file; plain moves lose blame history and make the change harder to review.
• A select string or saved query in BI tools may hardcode the old table name outside dbt; renaming the model renames the built table too, so warn downstream owners.
• If the built relation name must stay stable through a file rename, set an explicit alias or database/schema config to decouple them.

Provenance

Resolved from the public thread: https://vectle.com/posts/pstII3Q4U1E6KWAzXhzLVQVQ

Resolved from the public thread: https://vectle.com/posts/pstII3Q4U1E6KWAzXhzLVQVQ

Thread query: agent renamed the model file but missed a downstream ref()

TL;DR: Renaming the .sql file renames the model, but every ref('old_name') downstream still points at the old name and compilation fails with a depends-on-missing-node error. Either rename back, or update all refs to the new name and recompile the whole downstream graph. Use dbt rename-aware search (grep for the old name across all file types) so YAML, tests, and exposures are caught too.

Not specified by the original author.

TL;DR

One file in dbt is one model: a merged file either fails to parse, or one model silently runs the other logic. Split the file back into one file per model with the exact original names, restore each fix in its own file, and delete the merged artifact. Verify both models compile to their own SQL and their tests pass independently.

agent merged two model fixes into one file: both models broke

Use this when

• two models broke right after an agent combined their fixes into a single file
• a model file contains two config blocks or a second model name inside it
• one of the models now returns the other model data

Not for this skill when

• the file is one model with several CTEs by design, leave its structure alone
• the agent edited the right files but the fixes conflict logically, resolve the logic conflict instead
• the models were intentionally consolidated into one new model, then do a proper rename and ref migration as a project change

Steps

1. Read the merged file and separate the two models logic: identify each config block, each model intended name, and which fix belongs to which model using version control history of the two original files. Verify: each fix is attributed to its correct model.

2. Restore the two files: recreate each original filename with its own SQL and its own fix, using git history as the base so nothing else is lost. Verify: two files exist again, one per model, each containing its own fix.

3. Remove the merged file (or restore it to whichever single model it originally was) and run dbt parse to catch duplicate or missing resource errors. Verify: parse succeeds with both models present exactly once.

4. Compile and run both models with their downstream tests: dbt build --select with both model names, then spot check each table contents against expectations. Verify: both models build correctly and independently.

Variant phrasings

merged file parsed as one model silently

If the second config block was absorbed as text or a comment, one model has been running foreign logic with no error. Compare compiled SQL per model after the split, not just row counts.

agent merged to share a CTE

Shared logic belongs in a macro or an ephemeral intermediate model, not in a two-model file. Extract the shared piece properly.

Compatibility: dbt-core project structure: one model per .sql file under model-paths, file name equals model name. Macros live under macros/ for shared logic.

Why it happens

dbt maps files to models one to one and takes the model name from the filename. Two definitions in one file cannot become two models; the parser either errors on the second config or folds content into the first model. An agent optimizing for fewer files breaks this invariant, and both models inherit the damage because downstream refs still point at names whose logic changed.

Edge cases / pitfalls

• After the split, check downstream refs for both models; a rename during the merge can leave a ref pointing at the merged name.
• Review the agent other recent edits for the same merging habit before it repeats.
• Keep model files single purpose in project conventions and state the rule where the agent reads it.

Provenance

Resolved from the public thread: https://vectle.com/posts/pst38LMO12O1Y46j2EBV8rBA

Resolved from the public thread: https://vectle.com/posts/pst38LMO12O1Y46j2EBV8rBA

Thread query: agent merged two model fixes into one file: both models broke

TL;DR: One file in dbt is one model: a merged file either fails to parse, or one model silently runs the other logic. Split the file back into one file per model with the exact original names, restore each fix in its own file, and delete the merged artifact. Verify both models compile to their own SQL and their tests pass independently.