great expectations suite failed: expect_column_mean_to_be_between error out of range
An out-of-range mean is a distribution signal: compute the mean, median, and percentiles for the column directly and compare them with last week's values to see whether the whole distribution moved or a few extreme rows dragged the mean. Use when a Great Expectations suite using expect_column_mean_to_be_between. Not for a row-level expectation failure, where individual values rather than the aggregate are out of.
TL;DR
An out-of-range mean is a distribution signal: compute the mean, median, and percentiles for the column directly and compare them with last week's values to see whether the whole distribution moved or a few extreme rows dragged the mean. Fix the extreme rows or the shifted source; widening the range without that check just moves the alarm.
great expectations suite failed: expect_column_mean_to_be_between error out of rangeUse this when
- The check or run fails with this symptom: expectcolumnmeantobe_between fails with the observed mean outside the range.
- You are working on a Great Expectations suite using expectcolumnmeantobe_between and the failure repeats on the same data or configuration.
- You need the failing rows, rule, or stage isolated before changing thresholds or code.
Not for this skill when
- A row-level expectation failure, where individual values rather than the aggregate are out of bounds instead.
- The error text names a different tool, stage, or root cause. Match the verbatim error first and use that skill.
Steps
- Read the observed mean from the validation result and compute the mean, median, and 99th percentile directly in the warehouse for the same batch. Verify: the observed mean is confirmed and the distribution shape is visible
- Compare the same statistics for the previous good batch to find whether the shift is gradual or a single-batch jump. Verify: the change is classified as gradual drift or a sudden jump
- Inspect the extreme rows behind a jump, checking for unit changes, test records, or a duplicated load inflating the values. Verify: the rows driving the mean are identified
- Fix the driver: remove test or duplicate rows, correct the unit conversion, or reload the bad batch; only adjust the expected range when the new level is a verified steady state. Verify: the batch statistics return to a defensible level
- Rerun the suite and confirm the mean expectation and the related distribution expectations pass together. Verify: the aggregate expectations are green on the corrected batch
Variant phrasings
expectcolumnmeantobe_between out of range
The verbatim out-of-range phrasing.
great expectations column mean shifted outliers
The outlier phrasing, focused on percentiles versus the mean.
column mean expectation failing after load change
The load-change phrasing for unit or duplication problems.
Compatibility: Great Expectations 0.18 and 1.x column aggregate expectations
Why it happens
The mean is sensitive to outliers and unit changes, so a small number of extreme or mis-scaled rows can push it outside a range written for clean data. The expectation reports only the aggregate, which hides whether the bulk of the data moved or a tail appeared.
Edge cases / pitfalls
- A mean inside the range can still hide a changed distribution. Pair the mean check with a stdev or quantile expectation.
- Currency or unit migrations need a coordinated range update. Version the expectation change with the migration.
Provenance
Resolved from the public thread: https://vectle.com/posts/pst_EgD9TWQubuHs1q-C1f6sYQ