setting up dbt CI to catch breaking changes

TL;DR

Run dbt in CI on every pull request against production data, but only rebuild what changed. Slim CI with state selection and deferral gives you a real production-shaped check in minutes instead of rebuilding the world. Require it green before merge and breaking changes stop reaching main, which is the entire point.

setting up dbt CI to catch breaking changes

Use this when

• broken models keep getting merged to main
• you want pull request checks that actually test dbt changes
• full production rebuilds in CI take too long to be useful

Not for this skill when

• you have no CI system at all, set that up first
• the breakage is in source data, not in model code
• you need help writing the tests themselves, see the tests skill

Steps

1. Make the production manifest available in CI as the comparison baseline:

# fetch manifest.json from your last production run into ./prod-artifacts/
ls prod-artifacts/manifest.json

Expected output: the file exists in the CI workspace. State selection compares the PR code against this manifest to find exactly what changed.

1. Build only changed models and their children, deferring everything else to production:

dbt build --select state:modified+ --defer --state ./prod-artifacts

Expected output: changed models build in a CI schema while unselected refs resolve to production data. Fast and production-shaped at the same time.

1. Enforce contracts on critical models so shape changes fail loudly in the PR:

models:
  - name: fct_orders
    config:
      contract: {enforced: true}

Expected output: a PR that drops or renames a column fails CI instead of failing dashboards next week. The contract turns a silent break into a loud one.

1. Add a not-empty guard so a change that wipes a model cannot slip through:

-- tests/assert_fct_orders_not_empty.sql
SELECT 1 WHERE (SELECT COUNT(*) FROM {{ ref('fct_orders') }}) = 0

Expected output: the test fails if the CI build produced zero rows. An empty mart is never correct, and this catches the catastrophic case cheaply.

1. Wire it into pull requests with a workflow file and require it before merge:

# .github/workflows/dbt-ci.yml
name: dbt CI
on: [pull_request]
jobs:
  dbt:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - run: pip install dbt-postgres
      - run: dbt deps
      - run: dbt build --select state:modified+ --defer --state ./prod-artifacts

Expected output: every PR gets a dbt check that must pass before merge. Breaking changes get caught here, where the author is still around to fix them, not in production.

Variant phrasings

dbt slim CI setup

Steps 1 and 2 are slim CI in full: state selection plus deferral. It is the difference between a five minute check developers wait for and a two hour rebuild they merge around.

dbt CI on pull request with github actions

Step 5 is the minimal workflow. Add the production manifest download as the step right before the build, and keep credentials in encrypted env vars.

catching breaking dbt changes before merge

Contracts plus slim CI plus required status checks. Any one alone leaks, together they hold, which is why this skill teaches all three.

Why it happens

Without CI, the first test of a model change is the production run, which is also the first time stakeholders see it break. CI moves that first test to the pull request, where the author is present and the blast radius is zero. State selection makes it fast enough that developers actually wait for green instead of merging blind, which is the behavioral part most teams miss.

Edge cases

• The prod manifest must be fresh. A stale baseline makes state selection miss changes or flag everything, both of which erode trust.
• Deferral queries production data, which costs warehouse compute on every PR. Budget for it instead of being surprised.
• Credentials in CI belong in encrypted env vars, never in the repo. A leaked warehouse password is worse than no CI.
• Flaky tests train developers to ignore red CI. Fix or quarantine flaky tests before you require green builds.

Provenance

Resolved from the public thread: https://vectle.com/posts/pstF3NPPrXHUuCsNUqhWPkrQ