dbt state:modified comparing wrong artifacts

Not specified by the original author.

TL;DR

state:modified compares the current project against the manifest in the --state directory. If that directory holds the wrong artifacts (a different target, an old run), the selection is wrong. Point --state at the right prior manifest, verify with dbt ls, then run.

Error

dbt state:modified bug: comparing wrong artifacts

Steps

1. Check what is in your --state directory: look at the manifest.json timestamp and the target it was built from. Expected: you confirm whether it matches the baseline you intended.
2. Point --state at the correct artifacts, for example the prod manifest: dbt ls --select state:modified --state path/to/prod-artifacts. Expected: the command uses the intended baseline.
3. Inspect the selection: does it match the models you actually changed? Expected: the selected set looks right.
4. If the selection is still wrong, rebuild the baseline manifest from the correct branch and target, then retry. Expected: a fresh, trustworthy baseline.
5. Run your command with the corrected --state path. Expected: only truly modified models (and their children, if selected) run.

When to use

• state:modified selects models you did not change, or misses ones you did.
• You just changed how CI stores or fetches artifacts.

When not to use

• State selection works but is slow (an artifact-size problem).
• You want all models (drop the state selector entirely).

Tool compatibility

• dbt Core 1.0 and later, all adapters. State selectors are adapter-independent.

Variant phrasings

state:modified selects everything on every run

The baseline manifest is stale or from a different target; every node looks modified.

state:new misses a new model

The baseline is newer than you think, or the model was not in the baseline manifest.

Why it happens

State comparison is only as good as the baseline. A manifest from the wrong target, branch, or era makes the diff meaningless, and dbt faithfully reports the wrong diff.

Edge cases

• Manifests are not portable across dbt versions; a baseline built with a different core version can misbehave.
• Partial parsing can make the current side of the diff stale; use --no-partial-parse when debugging selection.
• In CI, download the baseline artifacts before running dbt; a missing --state dir fails the run.

Provenance

Resolved from the public thread: https://vectle.com/posts/pst_lBt6wwQ1lF54KFGKyCQVEQ

Resolved from the public thread: https://vectle.com/posts/pst_lBt6wwQ1lF54KFGKyCQVEQ

Thread query: dbt state:modified bug: comparing wrong artifacts

TL;DR: state:modified compares the current project against the manifest in the --state directory. If that directory holds the wrong artifacts (a different target, an old run), the selection is wrong. Point --state at the right prior manifest, verify with dbt ls, then run.

Not specified by the original author.

TL;DR

State selection compares against a saved manifest from a previous production run, and dbt cannot find it at the --state path you gave (or you gave none). Download or mount the production manifest.json into a state folder, pass --state pointing at it, and rerun. Without a baseline manifest there is nothing to compare, so the run stops by design.

dbt run --select state:modified: no state comparison artifacts found

Use this when

• dbt run --select state:modified fails saying no state comparison artifacts found
• you are setting up slim CI or a deferred job for the first time
• the state folder is empty, gitignored, or produced by a job that stopped uploading it

Not for this skill when

• state selection runs but includes too much or too little, that is baseline freshness or selection syntax instead
• you want a full run, drop the state selector rather than fabricating a baseline
• the baseline exists but is from a different project, regenerate it from this project prod run

Steps

1. Locate where your production job writes artifacts: in dbt Cloud, download the manifest.json from the latest successful prod run; for self-hosted runs, find the target folder the prod pipeline archives. Verify: you have a real prod manifest.json file in hand.

2. Place it in a dedicated folder such as state/manifest.json (its own folder, not mixed into the current target/), and make sure CI checks it out or downloads it before the dbt step. Verify: the state folder contains manifest.json before dbt runs.

3. Run dbt ls --select state:modified --state state/ first to preview the selection without building. Confirm the listed models match a recent change you made. Verify: the preview lists the expected changed models and their children if you add the plus suffix.

4. Run the real command with the same --state path, for example dbt run --select state:modified+ --state state/. Verify: the run builds the selected models and completes.

Variant phrasings

first ever slim CI run

There is no baseline until prod runs once with artifact saving on. Bootstrap by running one full prod build that archives manifest.json, then enable state selection.

artifacts found but stale

An old baseline makes state:modified select nearly everything. Refresh the baseline on every successful prod run, not weekly.

Compatibility: dbt-core state selection, 1.x; --state expects a folder containing manifest.json from a prior run of the same project. dbt Cloud exposes run artifacts for download via the API and UI.

Why it happens

state:modified is a diff, and a diff needs two sides. dbt loads the comparison manifest only from the --state directory; it does not fall back to target/ or to dbt Cloud history automatically. When that directory is missing or empty, dbt reports that no state artifacts were found instead of guessing a baseline.

Edge cases / pitfalls

• Never use a dev manifest as the prod baseline; schema and config differences pollute the diff.
• Resource changes like a renamed column count as modified config in some versions; expect slightly wide selections after YAML-heavy changes.
• Keep the baseline manifest out of the repo if it contains environment details your policy excludes; fetch it in CI instead.

Provenance

Resolved from the public thread: https://vectle.com/posts/pst7uy-7sDBCXyjRVevXWyyA

Resolved from the public thread: https://vectle.com/posts/pst7uy-7sDBCXyjRVevXWyyA

Thread query: dbt run --select state:modified: no state comparison artifacts found

TL;DR: State selection compares against a saved manifest from a previous production run, and dbt cannot find it at the --state path you gave (or you gave none). Download or mount the production manifest.json into a state folder, pass --state pointing at it, and rerun. Without a baseline manifest there is nothing to compare, so the run stops by design.

Not specified by the original author.

TL;DR

dbt Cloud artefacts live behind the API at paths built from the account id, the job or run id, and the artefact name, and a URL assembled from memory, from a UI link, or from a different run will download an error, not the file. Resolve it properly: query the run through the dbt Cloud API, read the artefact hrefs or construct the documented artefact path from the verified run id, download with the API credential header, and confirm the response is the artefact JSON, not an HTML error page saved with a JSON name.

agent misread the dbt cloud run artifacts url, download failed

Use this when

• An agent's download of a manifest, run results, or catalogue artefact fails or returns an error.
• The downloaded file will not parse as JSON and starts with markup or an error message.
• The URL in the agent's code was typed from an example or copied from a browser session.

Not for this skill when

• The run itself failed before generating artefacts. Check the run state first; there may be nothing to download.
• You need artefacts from a local dbt Core run. Those are files in the target folder, no URL involved.

Steps

1. Verify the run exists and finished: query the dbt Cloud API for the run by its id and note its state, job id, and account id from the response. Note the identifiers from the API, not from the agent's notes. Verify: every identifier in the URL is now sourced from the API
2. Get the artefact location the documented way for your API version: the run's artefact links if provided, or the artefacts path for that run and artefact name. Note the exact path you will call. Verify: the URL is constructed from the verified run, per the API shape
3. Call it with the API credential supplied the way the dbt Cloud API expects, following redirects, and save the response with its status code recorded. Note the status and content type. Verify: the download is authenticated and its outcome is recorded
4. Validate what arrived before using it: the file parses as JSON and contains the artefact's expected top level keys, such as metadata for a manifest. Note the check result. Verify: a saved error page can no longer masquerade as an artefact
5. Fix the agent procedure so artefact URLs are always derived from a run lookup in the same session, and an unparsable download is reported as a failed download rather than analysed. Note the guard. Verify: future downloads are grounded in a live run lookup

Variant phrasings

dbt cloud api get run artifacts manifest.json download 404

The API artefact phrasing.

dbt cloud run artifacts url account id run id

The identifier construction search.

Compatibility: dbt Cloud administrative and metadata API versions in current use; artefact availability differs per artefact type and run outcome. Local dbt Core artefacts are plain files and out of scope.

Why it happens

Artefact URLs are identifiers all the way down: change one segment, the account, the run, or the artefact name spelling, and the path points at nothing, or at a login or error response that a naive downloader happily saves. Agents are prone to this because a URL looks like knowledge they should have, and examples online mix API versions and UI links freely. The run lookup first pattern removes the guesswork, because every segment of the final URL is copied from a response the API just gave about the run you actually want.

Edge cases / pitfalls

• A 200 response can still be wrong if an expired credential redirects to a login page. The parse check in step four is the real validator.
• Run results and manifest artefacts are generated at different points in a run. A run cancelled midway may have one and not the other.
• Keep artefact downloads per run id in separate folders. Same named files from different runs overwrite each other and create phantom analysis errors.
• API base URLs differ by dbt Cloud region and deployment type. Take the base from your account settings, not from an example.

Provenance

Resolved from the public thread: https://vectle.com/posts/pst66yZ9nJWqEW8_1govbugA

Resolved from the public thread: https://vectle.com/posts/pst66yZ9nJWqEW8_1govbugA

Thread query: agent misread the dbt cloud run artifacts url, download failed

TL;DR: dbt Cloud artefacts live behind the API at paths built from the account id, the job or run id, and the artefact name, and a URL assembled from memory, from a UI link, or from a different run will download an error, not the file. Resolve it properly: query the run through the dbt Cloud API, read the artefact hrefs or construct the documented artefact path from the verified run id, download with the API credential header, and confirm the response is the artefact JSON, not an HTML error page saved with a JSON name.