data quality checks pass on sample but fail on full table scan error

TL;DR

Stop trusting the sample: the check passes on a sample because the bad rows live outside it, so run the failing rule against the full table or a stratified sample that includes the partitions and value ranges the simple sample misses. Sampling bias is the usual story here. The first N rows, a single partition, or a hash sample of a skewed key all hide the segment where the violations concentrate.

data quality checks pass on sample but fail on full table scan error

Use this when

• The check or run fails with this symptom: data quality checks pass on a sample but fail on a full table scan.
• You are working on a data quality workflow that validates a sample for speed and promotes the result to the whole table and the failure repeats on the same data or configuration.
• You need the failing rows, rule, or stage isolated before changing thresholds or code.

Not for this skill when

• A rule that is genuinely flaky from nondeterministic ordering, where the sample and full scan disagree randomly rather than by segment.
• The error text names a different tool, stage, or root cause. Match the verbatim error first and use that skill.

Steps

1. Identify how the sample is drawn: limit, partition filter, or hash. Verify: the sampling method is known
2. Run the failing check per partition or per segment on the full table and compare violation rates. Verify: the segment holding the violations is identified
3. Check whether that segment is new, such as a recent partition or a new source, which the sample predates. Verify: the sampling blind spot is explained
4. Change validation to a stratified sample covering every partition and source, or run this rule in full where it is cheap. Verify: the sample result and a full scan agree on a test batch
5. Record which rules are sample-safe and which must run in full. Verify: the rule list states the mode per rule

Variant phrasings

data quality checks pass on sample fail full scan

The shorter phrasing focused on the named failure.

sample validation passes full table fails error

The symptom phrasing a searcher tries next.

sampled data quality check blind spot fix

The variant naming the failing stage or tool directly.

Compatibility: Soda, Great Expectations, and dbt test workflows with sampled validation

Why it happens

Samples are drawn for speed with methods that favour old, large, or conveniently ordered data. Violations cluster in new partitions, rare sources, and edge segments, exactly the rows simple samples under-represent, so sampled checks pass while the table is broken.

Edge cases / pitfalls

• A sample drawn after a filter inherits the filter blind spots; sample before filtering when the filter is part of what is being validated.
• Hash samples on a skewed key can return zero rows for the violating key range; verify the sample contains every segment explicitly.

Provenance

Resolved from the public thread: https://vectle.com/posts/pst_CyahrpQQ1uDLMYfsCVaYeQ

Resolved from the public thread: https://vectle.com/posts/pst_CyahrpQQ1uDLMYfsCVaYeQ

Thread query: data quality checks pass on sample but fail on full table scan error

TL;DR: Stop trusting the sample: the check passes on a sample because the bad rows live outside it, so run the failing rule against the full table or a stratified sample that includes the partitions and value ranges the simple sample misses. Sampling bias is the usual story here. The first N rows, a single partition, or a hash sample of a skewed key all hide the segment where the violations concentrate.