dbt state selection in CI slim

TL;DR

Slim CI uses dbt's state selection to build and test only what changed: dbt compares the current project against a saved manifest from main (state:modified+) and runs just those models plus their children. CI goes from 40 minutes to 4. The trick is reliably fetching the baseline manifest artifact in CI; without it, state selection silently selects nothing or everything.

The query

dbt state selection in CI slim

Use this when

• dbt CI builds the whole project on every pull request and is too slow
• Setting up slim CI with GitHub Actions or dbt Cloud
• Developers avoid opening PRs because CI takes forever

Not for

• The first build of a project (nothing to compare against)
• Production scheduled runs (build everything)
• Debugging a broken baseline manifest

Steps

1. Produce a baseline manifest from the main branch. In CI, check out main (or download the last successful main artifact) and make sure target/manifest.json from that run is available as the comparison state.

Expected output: a manifest.json from main, accessible in the PR job.

1. Run dbt with state selection against the baseline:

dbt build --select state:modified+ --state ./baseline-target

Expected output: only changed models, their downstream children, and affected tests are selected.

1. Verify the selection is sane before trusting it. Run dbt ls with the same selector and eyeball the list: it should include your changed models plus downstream dependents. If it selects everything, the baseline state path is wrong.

Expected output: a short, sensible selection list, not the whole project.

1. Handle the edge cases: new models with no baseline entry are selected as modified, which is correct. Deleted models can break the comparison; regenerate the baseline after merges. Log the selected node count in CI so regressions in selection are visible.

Expected output: CI logs show a small stable selection per PR, and full builds still run on main.

Provenance

Resolved from the public thread: https://vectle.com/posts/pst4Lg4KZrPMsKwIda9KBAHA