dbt source freshness check stuck in warn state
Fixes dbt source freshness warnings by comparing the source's actual load time against the warn and error thresholds, then adjusting the thresholds or fixing the upstream pipeline. Use when dbt source freshness stays in warn. Not for freshness errors caused by a missing loaded_at_field.
TL;DR
Freshness compares the source's latest loaded timestamp against your warn_after and error_after thresholds. If the source genuinely loads slower than the thresholds allow, either relax the thresholds to match reality or fix the upstream pipeline. Then rerun dbt source freshness.
Error
dbt source freshness check stuck in warn stateSteps
- Run
dbt source freshnessand note the reported max loaded time and the age. Expected: you see how stale the source actually is. - Open the source definition and read the
freshnessblock:warn_afteranderror_aftervalues plus theloaded_at_field. Expected: you know the thresholds being applied. - Check the real upstream schedule: how often does the source actually refresh? Expected: you learn whether the threshold or the pipeline is wrong.
- If the pipeline is healthy but slower than the threshold, update
warn_afterto match reality. Expected: the config reflects the true SLA. - If the pipeline is late, fix the upstream job, then rerun
dbt source freshness. Expected: the status returns to pass.
When to use
dbt source freshnessreports warn persistently.- A source's SLA changed and the thresholds were never updated.
When not to use
- Freshness errors because
loaded_at_fieldis missing or misconfigured (fix the field first). - The source table itself is missing (a source definition problem).
Tool compatibility
- dbt Core 1.0 and later, all adapters. Freshness SQL is adapter-specific under the hood.
Variant phrasings
Freshness flips between warn and pass
The source loads right around the threshold; widen the threshold or speed the pipeline.
Freshness error_after firing on weekends
The pipeline does not run on weekends; scope thresholds or pause checks accordingly.
Why it happens
Freshness is a simple age comparison. Thresholds written when the pipeline was fast go stale as data volumes grow, and the check correctly reports that the source is older than allowed.
Edge cases
loaded_at_fieldmust be a timestamp the warehouse can max over; a string column gives nonsense ages.- Timezone mismatches between the loaded field and the check can shift ages by hours.
- Snapshotted sources report the snapshot's freshness, not the raw table's; check which one you configured.
Provenance
Resolved from the public thread: https://vectle.com/posts/pst_HtSb6fVadSwCjuftl3sBKg