soda scan failed error "anomaly detection requires at least 30 historic measurements"
Give the anomaly detector its history: Soda anomaly detection needs at least 30 historic measurements for a metric, so let the check collect measurements over time, or seed the history from past scan results, before expecting anomaly verdicts. Use when a Soda Core anomaly detection check on a metric that is new or newly renamed. Not for a metric whose history exists under a different check identity after a rename,
TL;DR
Give the anomaly detector its history: Soda anomaly detection needs at least 30 historic measurements for a metric, so let the check collect measurements over time, or seed the history from past scan results, before expecting anomaly verdicts. Thirty measurements is the floor for a baseline the detector trusts. A new check has days, not a baseline, and Soda refuses to guess rather than firing on noise.
soda scan failed error "anomaly detection requires at least 30 historic measurements"Use this when
- The check or run fails with this symptom: a Soda scan fails with anomaly detection requires at least 30 historic measurements.
- You are working on a Soda Core anomaly detection check on a metric that is new or newly renamed and the failure repeats on the same data or configuration.
- You need the failing rows, rule, or stage isolated before changing thresholds or code.
Not for this skill when
- A metric whose history exists under a different check identity after a rename, which needs the identity aligned rather than thirty new days.
- The error text names a different tool, stage, or root cause. Match the verbatim error first and use that skill.
Steps
- Check how many measurements Soda has stored for the metric. Verify: the stored count is below thirty
- Confirm the scan has run on a regular schedule and results are being stored. Verify: measurements accumulate per run, or storage is off and nothing accumulates
- If the check was renamed, align its identity with the previous check so the existing history attaches. Verify: the stored count jumps over the floor
- Otherwise run the scan on schedule until the floor is reached, keeping the check in warn mode meanwhile. Verify: the measurement count reaches thirty
- Switch the check to fail mode and run the scan. Verify: anomaly detection evaluates against the collected baseline
Variant phrasings
soda anomaly detection requires 30 historic measurements
The shorter phrasing focused on the named failure.
soda scan failed anomaly historic measurements
The symptom phrasing a searcher tries next.
soda anomaly detection history floor fix
The variant naming the failing stage or tool directly.
Compatibility: Soda Core 3.x anomaly detection checks and Soda Cloud measurement history
Why it happens
Anomaly detection compares today with a learned history. New checks, renamed checks, and scans that do not store results all present as zero history, and Soda treats a verdict without a baseline as worse than no verdict, so it fails the configuration instead.
Edge cases / pitfalls
- Backfilling history from a burst of same-day scans defeats the purpose; measurements need real time spread to form a baseline.
- Renaming a check for clarity resets its anomaly history unless the identity is migrated; plan renames with that cost stated.
Provenance
Resolved from the public thread: https://vectle.com/posts/pst_RZ8B0yNRllx0gnuCXf-vxw