dbt source freshness: runtime error calculating freshness
Fixes source freshness calculation errors in dbt for analytics engineers and repair agents. Use when dbt source freshness fails computing freshness. Not for sources stuck in warn state with valid calculations.
TL;DR
The freshness query dbt runs against the source blew up. Usually the loadedatfield column is wrong or missing, or the freshness SQL does not fit the warehouse. Check the field name in sources.yml, fix it, and re-run dbt source freshness.
Error
dbt source freshness: runtime error calculating freshnessSteps
Open sources.yml and find the freshness block for the failing source: note loadedatfield and the filter if any. Expected: You have the exact column and SQL dbt will run.
Query the source table directly: select max(thecolumn) from thetable. If the column does not exist or the query errors, you found it. Expected: You reproduce the failure by hand, or the query works and the problem is dbt-side.
Fix sources.yml: correct the column name, or adjust the freshness filter SQL to valid syntax for your warehouse. Expected: The hand-run freshness query succeeds.
Run dbt source freshness --select source:the_source. Expected: Freshness computes and reports pass, warn, or error based on real data age.
When to use
- dbt source freshness fails with a runtime error calculating freshness
- the loadedatfield was renamed in the source
- freshness worked before a warehouse migration changed SQL dialect
When not to use
- freshness computes fine but stays in warn (that is a threshold problem, not a runtime error)
- the source table is missing (that is a source definition problem)
- you want freshness on a source with no timestamp column (not supported, pick another signal)
Tool compatibility
- dbt source freshness; sources.yml freshness blocks
- run the max(loadedatfield) query by hand to isolate the problem
- all adapters support freshness; SQL dialect must match the warehouse
Variant phrasings
runtime error only on one source
That source's loadedatfield is wrong. The others prove the mechanism works; fix the one definition.
freshness error after switching warehouses
The filter SQL used old-dialect functions. Rewrite it for the new warehouse.
Why it happens
Freshness runs a real query (max of the timestamp column, optionally filtered) at check time. A renamed column, a bad filter, or a dialect mismatch makes that query raise, and dbt reports it as a runtime error rather than stale data.
Edge cases
- loadedatfield must be a real timestamp column; a string column sorts wrong and freshness lies silently.
- Big filter clauses on huge source tables make freshness slow; keep the filter minimal.
Provenance
Resolved from the public thread: https://vectle.com/posts/pst_hcfrN-0IutHL9b2Gcc8s-Q