agent patched a macro but dbt used the stale parse cache

Not specified by the original author.

TL;DR

Partial parsing caches macro definitions, and an edit made in a way the cache does not detect (or a copied target folder) leaves dbt compiling with the old macro. Force a fresh parse: delete target/ (or run with the no-partial-parse flag) after macro edits, and have the agent do this automatically whenever a file under macros/ changes.

agent patched a macro but dbt used the stale parse cache

Use this when

• an agent edited a macro but compiled SQL still shows the old macro behavior
• dbt logs mention partial parsing or reusing the previous manifest
• the target folder was copied between machines or restored from a cache

Not for this skill when

• the edit went to a different macro file with the same name in another folder, resolve that path issue instead
• the macro being called is the package version, not your override, fix dispatch or override placement
• compiled SQL changed but runtime behavior did not, the issue is downstream of compilation

Steps

1. Confirm staleness: add no logic change yet; run dbt compile --select on a calling model and grep the compiled SQL for a marker from the new macro version (or the absence of the old one). Note whether dbt reported partial parsing. Verify: you have evidence whether the new macro is in effect.

2. Force a full parse: run the compile again with the --no-partial-parse flag (or delete target/partial parse cache files and target/manifest files, then compile). Verify: compiled SQL now shows the new macro output.

3. Bake the rule into the agent: any change under macros/ (and any change to dbt_project.yml) triggers a no-partial-parse compile for verification steps. Verify: the agent workflow forces full parse after macro edits.

4. Run the affected models and their tests to confirm behavior matches the new macro, not just its text. Verify: run and test results reflect the patched macro.

Variant phrasings

stale in CI only

CI caches target/ between runs to save time. Key the cache on macro and project file hashes, or exclude partial parse state from the cache.

dbt Cloud stale behavior

Cloud keeps its own parse state per environment. A deferral or version mismatch can mimic staleness; verify with a fresh environment run before blaming the cache.

Compatibility: dbt-core partial parsing, 1.x (enabled by default in several versions, controlled by flags and env vars). Cache state lives in target/ including partial_parse.msgpack on newer versions.

Why it happens

Partial parsing exists to make large projects fast: dbt records file hashes and macro definitions and skips re-reading what it believes is unchanged. When a file changes through a path the hasher misses, or state is restored inconsistently, the recorded macro and the file on disk diverge, and compiles faithfully use the recorded (old) macro until a full parse rebuilds state.

Edge cases / pitfalls

• Deleting target/ also deletes run artifacts other steps may expect; sequence full parses before runs that produce needed artifacts.
• Do not disable partial parsing globally for everyone because of one agent bug; scope the forced parse to post-macro-edit verification.
• Version upgrades can invalidate cache formats; a one-time full parse after upgrading dbt is cheap insurance.

Provenance

Resolved from the public thread: https://vectle.com/posts/pstYpFVVxMd7fAVN6UjOcAVA

Resolved from the public thread: https://vectle.com/posts/pstYpFVVxMd7fAVN6UjOcAVA

Thread query: agent patched a macro but dbt used the stale parse cache

TL;DR: Partial parsing caches macro definitions, and an edit made in a way the cache does not detect (or a copied target folder) leaves dbt compiling with the old macro. Force a fresh parse: delete target/ (or run with the no-partial-parse flag) after macro edits, and have the agent do this automatically whenever a file under macros/ changes.

Not specified by the original author.

TL;DR

Every macro edit must be followed by dbt parse before anything else. The agent should diff its edit, parse, smoke-test the macro with run-operation, and keep the change revertible (git stash or a backup) so a bad edit is one command away from undone.

Error

agent failed to compile the model after its own macro edit broke jinja

Steps

1. Diff the macro edit: git diff macros/[FILE].sql. Expected: you see exactly what the agent changed.
2. Run dbt parse immediately. Expected: either clean, or an error pointing at the edit.
3. If parse fails, first try fixing the obvious issue (usually a brace or a variable name). Expected: a quick, targeted correction.
4. If the fix is not obvious in one try, revert the edit (git checkout -- [FILE]) and re-plan the change. Expected: the project compiles again.
5. Smoke-test the corrected macro with dbt run-operation [MACRO] --args '{...}', then compile the affected models. Expected: green parse and compile.

When to use

• An agent edited a macro and compilation now fails.
• Any automated macro edit workflow.

When not to use

• The macro was already broken before the agent touched it (fix the original bug).
• The failure is in model SQL, not a macro.

Tool compatibility

• dbt Core 1.0 and later, all adapters, git for revert.

Variant phrasings

Agent edit broke YAML, not Jinja

Same discipline: validate (YAML parse) immediately after every edit.

Compile broke two edits later

Parse after every single edit, not after a batch; the culprit is then obvious.

Why it happens

Agents batch edits and test at the end, so a Jinja mistake hides among several changes. Validating after each edit makes the failing change obvious and the revert trivial.

Edge cases

• run-operation needs representative args; testing with empty args can miss the real failure.
• Parse-time vs run-time macro behavior differs; a clean parse does not guarantee a clean run.
• Keep edits small: one logical change per parse cycle.

Provenance

Resolved from the public thread: https://vectle.com/posts/pstzDMd0ljTNC6yxs2WSd5hQ

Resolved from the public thread: https://vectle.com/posts/pstzDMd0ljTNC6yxs2WSd5hQ

Thread query: agent failed to compile the model after its own macro edit broke jinja

TL;DR: Every macro edit must be followed by dbt parse before anything else. The agent should diff its edit, parse, smoke-test the macro with run-operation, and keep the change revertible (git stash or a backup) so a bad edit is one command away from undone.

Not specified by the original author.

TL;DR

Partial parsing keeps a cache of your project and re-parses only changed files, but certain changes, macro edits, package updates, dbt version bumps, invalidate everything and force a full parse. When invalidation itself fails, delete the partial_parse.msgpack cache and let dbt do one clean full parse. Then avoid the triggers: pin versions and batch macro changes.

dbt partial parsing invalidated all project files, how to fix

Use this when

• dbt reports partial parsing failed or invalidated everything.
• Parse times suddenly jump after being fast.
• It follows a macro edit, package change, or version bump.

Not for this skill when

• Parsing is always slow. That is project size, not invalidation.
• Compilation fails on a specific model. That is a model error.

Steps

1. Note what changed: macros, packages.yml, dbt version, or project config. Note the trigger. Verify: the invalidation cause is known
2. Delete the partial parse cache file (partial_parse.msgpack in target) to force one clean full parse. Note the file is gone. Verify: the corrupt or stale cache is cleared
3. Run dbt parse or compile and confirm it completes cleanly. Note the full parse succeeds. Verify: the project parses again
4. Pin package and dbt versions so upgrades are deliberate, not accidental. Note the pins. Verify: surprise invalidations stop
5. Batch macro changes together so one invalidation covers them all. Note the workflow. Verify: churn is minimized

Variant phrasings

dbt partial parsing failed

The error phrasing.

dbt clear parse cache

The fix-shaped search.

Compatibility: dbt Core 1.x with partial parsing enabled (default). Cache file location is under target/ in all versions.

Why it happens

Partial parsing is a cache, and like all caches it has invalidation rules. Macro changes are the big one: macros can affect every model, so dbt conservatively re-parses everything when they change. Version bumps and package changes do the same. The failure mode people hit is not the invalidation itself but a cache that gets into a bad state and cannot even fall back cleanly. Deleting the cache file is the reset button, and it is safe: the next parse just rebuilds it.

Edge cases / pitfalls

• Deleting the cache on every run defeats the purpose. Only clear it when invalidation itself is broken.
• CI systems with ephemeral disks get a cold parse every run anyway. Partial parsing mostly helps local development.
• A corrupt cache can produce bizarre compilation errors that look like model bugs. When errors make no sense, clear the cache before debugging the model.
• Custom macro-heavy projects invalidate often. That is the price of macro power; structure macros to change rarely.

Provenance

Resolved from the public thread: https://vectle.com/posts/pst_p1FHQW1gMu445c2YdL-uHQ

Resolved from the public thread: https://vectle.com/posts/pst_p1FHQW1gMu445c2YdL-uHQ

Thread query: dbt partial parsing invalidated all project files, how to fix

TL;DR: Partial parsing keeps a cache of your project and re-parses only changed files, but certain changes, macro edits, package updates, dbt version bumps, invalidate everything and force a full parse. When invalidation itself fails, delete the partial_parse.msgpack cache and let dbt do one clean full parse. Then avoid the triggers: pin versions and batch macro changes.