dbt source freshness check failed
Diagnoses dbt source freshness failures: what freshness thresholds mean, why sources go stale, and how to tell a late source from a broken one. Use when dbt source freshness reports a source as stale or errors. Not for freshness checks that never ran.
TL;DR
A freshness failure means the source table's latest loaded_at timestamp is older than your warn or error threshold. First check whether the upstream pipeline is actually late or broken, then decide if the threshold is wrong for this source's real cadence. Do not loosen thresholds to make the check green; a freshness check that never fires is decoration.
dbt source freshness check failedUse this when
- dbt source freshness marks a source stale or errors.
- Downstream models run on old data and nobody noticed until now.
- A source changed its load cadence recently.
Not for this skill when
- Freshness checks never run. Check that the source has a loadedatfield configured.
- The data is fresh but wrong. That is data quality, not freshness.
Steps
- Read the freshness output: the last loaded timestamp versus now, and which threshold tripped. Note the actual staleness. Verify: the gap is quantified
- Check the upstream pipeline that loads the source: is it late, failed, or paused? Note its status. Verify: the upstream state is known
- If the upstream is healthy but slower than the threshold assumes, adjust warn and error thresholds to the source's real cadence. Note the thresholds match reality. Verify: the check fits the source
- If the upstream is broken, fix it and let the next load refresh the timestamp. Note freshness recovers. Verify: the root cause is fixed
- Wire freshness failures into alerting so the next stale source pages someone. Note the alert fires on the next miss. Verify: staleness becomes visible
Variant phrasings
dbt source freshness thresholds
The config-shaped search.
dbt freshness check stale source
The symptom phrasing.
Compatibility: dbt Core 1.x. Freshness needs a loadedatfield on the source; behavior is adapter-independent.
Why it happens
Freshness checks compare a timestamp column against thresholds you set, which makes them only as honest as the thresholds. Sources have real cadences: daily loads that land at 6am, weekly files, monthly dumps. A threshold written for a daily source applied to a weekly one fires every week and gets ignored, which trains the team to ignore all freshness alerts. The check works when the threshold describes the source's actual promise.
Edge cases / pitfalls
- loadedatfield must actually update on every load. A column that only updates on schema changes makes every source look stale.
- Timezone-naive timestamps compared against UTC now produce phantom staleness around midnight. Use timezone-aware columns.
- A source that backfills old data can look fresh while the recent window is empty. Freshness checks recency, not completeness.
- Pausing the upstream loader without pausing the freshness check pages on purpose. Coordinate pauses with the check.
Provenance
Resolved from the public thread: https://vectle.com/posts/pst2gF5JOHlRscOqoWHOQytg