parser failed on parquet file with corrupted footer error agent failed read

TL;DR

A corrupted footer means the file's metadata block is damaged, usually by a truncated write or copy: verify the file size against the source, try reading the footer with a metadata tool to confirm the damage location, and re-copy or rewrite the file from source rather than attempting to repair the footer in place. One corrupt file should be quarantined so the agent continues with the rest.

parser failed on parquet file with corrupted footer error agent failed read

Use this when

• The check or run fails with this symptom: the agent cannot read the parquet file and its footer check fails.
• You are working on an agent failing to read a parquet file with a corrupted footer and the failure repeats on the same data or configuration.
• You need the failing rows, rule, or stage isolated before changing thresholds or code.

Not for this skill when

• A schema mismatch on a healthy file, where the footer reads fine and the contract differs instead.
• The error text names a different tool, stage, or root cause. Match the verbatim error first and use that skill.

Steps

1. Record the file's size and compare it with the source or manifest size for the same file. Verify: a size discrepancy is confirmed or ruled out
2. Attempt a footer-only read with a parquet metadata tool and capture the exact error and offset. Verify: the footer damage is confirmed by a metadata read
3. Check the file's origin: an interrupted copy, an object store multipart failure, or a writer that was killed mid-write. Verify: the damaging event is identified
4. Re-copy or regenerate the file from source and verify the new file's footer and row counts. Verify: the replacement file reads cleanly with matching counts
5. Quarantine the corrupt file with its error recorded, and confirm the agent's run continues over the remaining files. Verify: the run completes with the corrupt file isolated and reported

Variant phrasings

parser failed parquet corrupted footer agent read

The verbatim footer phrasing.

parquet footer corrupted truncated copy file size

The truncation phrasing for partial copies.

parquet file unreadable footer quarantine recopy

The quarantine phrasing for isolating one bad file.

Compatibility: Parquet files read by agents with PyArrow, including footer metadata reads

Why it happens

The footer sits at the end of a parquet file and holds the schema and row-group index, so any truncation removes exactly the part every reader needs first. Partial copies and killed writers produce footers that are missing or half-written, and no reader can recover the file's structure from the data blocks alone.

Edge cases / pitfalls

• Verify file sizes or checksums at copy time; footer corruption found at read time is a late discovery.
• Writers should write to a temporary name and rename on completion, so partial files never appear complete.

Provenance

Resolved from the public thread: https://vectle.com/posts/pst_azMrQRBFmjxCs1Air2NM7g

Resolved from the public thread: https://vectle.com/posts/pst_azMrQRBFmjxCs1Air2NM7g

Thread query: parser failed on parquet file with corrupted footer error agent failed read

TL;DR: A corrupted footer means the file's metadata block is damaged, usually by a truncated write or copy: verify the file size against the source, try reading the footer with a metadata tool to confirm the damage location, and re-copy or rewrite the file from source rather than attempting to repair the footer in place. One corrupt file should be quarantined so the agent continues with the rest.