elementary data quality test failed error "table anomalies: row count anomaly detected"

Not specified by the original author.

TL;DR

Do not widen the anomaly bounds to silence it: open the Elementary test result, read the observed row count against the training window average and standard deviation, and check whether a real load change explains the jump before touching the test configuration. Elementary flags a table anomaly when the latest metric falls outside the bounds learned from historical values, so a genuine partial load and a stale training window look identical in the alert.

elementary data quality test failed error "table anomalies: row count anomaly detected"

Use this when

• The check or run fails with this symptom: Elementary reports table anomalies with a row count anomaly detected message.
• You are working on an Elementary-monitored dbt project where a table-level anomaly test fires and the failure repeats on the same data or configuration.
• You need the failing rows, rule, or stage isolated before changing thresholds or code.

Not for this skill when

• A source table whose volume legitimately changed after a backfill or a new upstream feed, where the fix is refreshing the training window rather than the pipeline.
• The error text names a different tool, stage, or root cause. Match the verbatim error first and use that skill.

Steps

1. Open the Elementary result for the failing test and note the metric value, the expected range, and the timestamps of the training window. Verify: the observed value and its bounds are recorded, not just the fail status
2. Query the source table row count per load for the same window and compare it with the Elementary metric series. Verify: a matching dip or spike in the source confirms a real load change
3. Check for a partial or duplicate load on the flagged day in the pipeline run history. Verify: the load event that explains the anomaly is identified or ruled out
4. If the load was real and correct, refresh the Elementary training window or set the anomaly sensitivity for that table; if the load was partial, rerun the load instead. Verify: the chosen fix matches the confirmed cause
5. Rerun the Elementary tests for that table. Verify: the table anomalies test passes on the corrected data or the refreshed window

Variant phrasings

elementary table anomalies row count anomaly detected

The shorter phrasing focused on the named failure.

elementary data quality test failed row count

The symptom phrasing a searcher tries next.

dbt elementary anomaly test false positive row count

The variant naming the failing stage or tool directly.

Compatibility: Elementary 0.10 and later with dbt 1.5 and later, warehouse-native Elementary tests

Why it happens

Elementary learns normal metric ranges from a trailing window of past values. A partial load, a duplicate load, or a window polluted by an earlier incident moves the observed value outside the learned bounds even when the test itself is configured correctly.

Edge cases / pitfalls

• A training window that includes a previous incident teaches the test that the incident was normal; exclude known bad days instead of widening sensitivity for the whole table.
• Row count anomalies on tables fed by multiple sources need per-source counts first; the aggregate hides which feed moved.

Provenance

Resolved from the public thread: https://vectle.com/posts/pstDTBY6HUUvFx2iGgq8uHRQ

Resolved from the public thread: https://vectle.com/posts/pstDTBY6HUUvFx2iGgq8uHRQ

Thread query: elementary data quality test failed error "table anomalies: row count anomaly detected"

TL;DR: Do not widen the anomaly bounds to silence it: open the Elementary test result, read the observed row count against the training window average and standard deviation, and check whether a real load change explains the jump before touching the test configuration. Elementary flags a table anomaly when the latest metric falls outside the bounds learned from historical values, so a genuine partial load and a stale training window look identical in the alert.

Not specified by the original author.

TL;DR

An Airbyte row count anomaly usually sits in the sync, not the check: open the sync history for the affected stream, compare source and destination counts per stream, and look for a reset, a schema change, or a cursor that stopped advancing. Fix the sync configuration or run a targeted backfill for the stream, then let the check re-evaluate.

how to fix row count anomaly failure error in airbyte sync data quality check

Use this when

• The check or run fails with this symptom: the data quality check on an Airbyte-synced table reports a row count anomaly.
• You are working on an Airbyte sync whose destination row counts trip the quality check and the failure repeats on the same data or configuration.
• You need the failing rows, rule, or stage isolated before changing thresholds or code.

Not for this skill when

• A check failing on a non-Airbyte pipeline, where sync history and cursors do not apply instead.
• The error text names a different tool, stage, or root cause. Match the verbatim error first and use that skill.

Steps

1. Open the Airbyte sync history for the stream and record attempts, rows emitted, and any resets in the anomaly window. Verify: the sync history for the window is documented
2. Compare per-stream source counts with destination counts to isolate whether one stream or the whole connection drifted. Verify: the anomaly is scoped to a stream or the connection
3. Inspect the stream's cursor field and schema for changes that would stall or duplicate extraction. Verify: cursor and schema state are checked
4. Fix the configuration, clear a stuck cursor deliberately, or run a backfill for the affected window, documenting any reset's impact. Verify: the sync is corrected or the backfill is running with a known scope
5. After the sync completes, reconcile counts again and rerun the quality check. Verify: counts reconcile and the check passes

Variant phrasings

row count anomaly airbyte sync data quality

The verbatim Airbyte anomaly phrasing.

airbyte stream row count mismatch destination

The reconciliation phrasing for per-stream counts.

airbyte cursor stuck row count check failed

The cursor phrasing for stalled extraction.

Compatibility: Airbyte syncs with destination data quality checks on row counts

Why it happens

Airbyte moves data stream by stream with cursors, so a stalled cursor, a stream reset, or a source schema change alters destination counts in ways that look like data anomalies downstream. The quality check sees the destination only; the sync history holds the explanation.

Edge cases / pitfalls

• Resets re-sync full history and can duplicate if the destination write mode is append. Confirm the write mode first.
• Alert per stream, not per connection. One sick stream hides inside a healthy connection total.

Provenance

Resolved from the public thread: https://vectle.com/posts/psta4YSjnBnCieIowbDVTiUw

Resolved from the public thread: https://vectle.com/posts/psta4YSjnBnCieIowbDVTiUw

Thread query: how to fix row count anomaly failure error in airbyte sync data quality check

TL;DR: An Airbyte row count anomaly usually sits in the sync, not the check: open the sync history for the affected stream, compare source and destination counts per stream, and look for a reset, a schema change, or a cursor that stopped advancing. Fix the sync configuration or run a targeted backfill for the stream, then let the check re-evaluate.

Not specified by the original author.

TL;DR

Verify the count query itself first (a wrong partition filter is the number one cause), then break the count down by partition, date, or source to find which slice vanished. The fix is almost always in the pipeline code: a changed filter, a join that started dropping rows, or an over-aggressive dedup step. Actual data loss at the source is the rarest cause, so check your own code before blaming upstream.

ALERT: orders_daily row count dropped 90% (1,000,000 -> 100,000)

Use this when

• A table lost most of its rows between runs
• A volume check fired on a sudden drop
• A recent deploy changed row counts

Not for this skill when

• Counts drift gradually over weeks (trend issue, not an incident)
• The schema changed (different root cause)
• One BI tool shows different counts than the warehouse (reporting bug)

Steps

1. Re-run the count manually with the exact same filters as the pipeline:

SELECT COUNT(*) FROM orders_daily WHERE dt = '2026-10-03';

Expected output: confirmation the drop is real and not a misread alert. Copy the filter from the pipeline code, dont retype it from memory.

2. Break the count down by load dimensions to find the missing slice:

SELECT dt, source_system, COUNT(*)
FROM orders_daily
WHERE dt BETWEEN '2026-09-26' AND '2026-10-03'
GROUP BY dt, source_system ORDER BY dt, source_system;

Expected output: the specific date, partition, or source that collapsed. A 90 percent drop concentrated in one source or one day narrows the search enormously.

3. Check whether the upstream extract actually finished:

# check the extract job logs for the affected date
# look for: partial files, early termination, zero-byte outputs

Expected output: the upstream job's status for the affected window. A partial extract that "succeeded" is a classic cause: the loader happily loaded half the data.

4. Diff recent pipeline code changes against the last good run:

git log --oneline -5 -- [pipeline path]
git diff [last good commit] HEAD -- [pipeline path] | grep -iE "where|join|filter|dedup"

Expected output: the recent change touching filters, joins, or dedup logic. New WHERE clauses and changed join types are the usual culprits.

5. Suspect dedup logic specifically: compare raw input counts against output counts:

SELECT COUNT(*) AS raw_rows, COUNT(DISTINCT order_id) AS distinct_ids
FROM staging_orders WHERE dt = '2026-10-03';

Expected output: if raw rows are fine but distinct IDs collapsed, the dedup key or logic changed. Over-aggressive dedup (wrong key, case changes, null handling) silently eats the majority of rows.

Variant phrasings

sudden row count drop etl

Follow the steps in order: verify, slice, check upstream, diff code, check dedup. Skipping straight to "the source lost data" wastes the most time because it is the least likely cause.

table lost rows after load

Check whether the load was a full refresh that replaced good data with a partial extract. If so, the fix is rerunning the extract completely, plus a guard (step: fail the load when input counts drop) so it cant happen again.

row count anomaly alert tuning

Alert on relative change per partition with a floor (ignore tiny tables), and require the drop to persist across two consecutive runs before paging. Single-run dips are often late data, not data loss.

Why it happens

Row counts are the cheapest data-quality signal, and sudden drops almost always trace to the pipeline's own logic: a filter added for one case that excludes the general case, an inner join that used to match, a dedup key that stopped being unique. The data rarely vanishes; it gets excluded. That is why slicing the count by dimensions finds the answer fast: the excluded slice has a pattern.

Edge cases

• Late-arriving data makes the most recent partition look low until it lands; compare against the same lag on previous days before panicking.
• Soft deletes versus hard deletes: a new soft-delete flag can make "active" counts drop while the rows still exist.
• Timezone shifts move rows across day boundaries; a drop on one day paired with a spike on the next is a timezone bug, not data loss.
• Sampling in the count query itself (TABLESAMPLE, approximate counts) can show phantom drops; verify with an exact count.
• Backfill overlaps can double-count instead of dropping; the same slicing technique diagnoses spikes too.

Provenance

Resolved from the public thread: https://vectle.com/posts/pstM9PWshAX3vIvktpbGk-vA

Resolved from the public thread: https://vectle.com/posts/pstM9PWshAX3vIvktpbGk-vA

Thread query: "row count dropped 90%" pipeline debugging

TL;DR: Verify the count query itself first (a wrong partition filter is the number one cause), then break the count down by partition, date, or source to find which slice vanished. The fix is almost always in the pipeline code: a changed filter, a join that started dropping rows, or an over-aggressive dedup step. Actual data loss at the source is the rarest cause, so check your own code before blaming upstream.