warehouse load failed error: bad records exceeded max bad records tolerance failure

TL;DR

Exceeding the bad-records tolerance stops the load to protect the table: pull the loader's error list, group the bad records by error type, and fix the dominant type at the producer, since tolerance breaches are almost never random. Raising the tolerance to pass the load imports the very rows the limit exists to stop.

warehouse load failed error: bad records exceeded max bad records tolerance failure

Use this when

• The check or run fails with this symptom: the load aborts with bad records above the configured maximum.
• You are working on a warehouse load failing because bad records exceeded the tolerance and the failure repeats on the same data or configuration.
• You need the failing rows, rule, or stage isolated before changing thresholds or code.

Not for this skill when

• A load failing on its first record with a connection or permission error, which is not a data tolerance issue instead.
• The error text names a different tool, stage, or root cause. Match the verbatim error first and use that skill.

Steps

1. Extract the loader's bad-record errors and group them by type and field. Verify: bad records are grouped by dominant error type
2. Sample records of the dominant type in the source and identify the format or content change behind them. Verify: the dominant type's source cause is found
3. Check whether a producer deploy, a new source file variant, or a delimiter or encoding change coincides with the breach. Verify: a coincident change is confirmed or ruled out
4. Fix the producer or the loader's parsing configuration for that type, keeping the tolerance unchanged. Verify: the dominant error type is eliminated at its source
5. Reload the batch and confirm bad records fall to zero or well under tolerance, with any remainder inspected. Verify: the reload completes inside tolerance with remainders explained

Variant phrasings

warehouse load bad records exceeded max tolerance

The verbatim tolerance phrasing.

load failed too many bad records error types

The grouping phrasing for dominant error types.

bad records tolerance producer format change load

The producer-change phrasing for systematic bad rows.

Compatibility: Warehouse loaders on BigQuery, Redshift, and Snowflake with bad-record tolerances

Why it happens

Loaders count malformed records against a tolerance so a few bad rows do not block a batch; breaching it means bad rows became systematic. A format change at the producer is the usual cause, and the grouped error list shows it immediately.

Edge cases / pitfalls

• Keep tolerance low and alert on any bad records, not only breaches. The trickle precedes the flood.
• Retain the rejected records with their errors. They are the producer's bug report.

Provenance

Resolved from the public thread: https://vectle.com/posts/pstTwaP2p0Yqi8rmSuNtGolg