S3 multipart upload failed mid-backup

TL;DR

A multipart upload that fails mid-backup leaves uploaded parts billed and orphaned until aborted, and the job fails on whichever part or completion call broke. List incomplete uploads on the bucket, abort the stale ones, fix the part failure cause such as network drops or size limits, add a bucket lifecycle rule to auto-abort future orphans, and rerun the backup.

S3 multipart upload failed mid-backup

Use this when

• Backup log reports multipart upload failed, upload aborted, or a part error.
• The bucket shows incomplete multipart uploads accumulating.
• Large backups fail while small ones using single requests succeed.

Not for this skill when

• All S3 requests time out before any part uploads. Path problem first.
• Access denied on the bucket. Permissions, not multipart mechanics.

Steps

1. List incomplete multipart uploads for the bucket and record their ages and total part storage. Verify: orphaned uploads are inventoried
2. Abort incomplete uploads older than your longest backup window. Verify: the incomplete list is clean
3. From the job log identify which part failed and why, such as entity too large, timeout, or connection reset. Verify: the part failure cause is named
4. Adjust part size within S3 limits so each part fits your link reliability, and confirm the tool retries parts. Verify: settings are within limits and retries are enabled
5. Add a lifecycle rule aborting incomplete multipart uploads after a few days and rerun the backup. Verify: the backup completes and no new orphans linger

Variant phrasings

s3 incomplete multipart upload cleanup backup

The cleanup phrasing.

multipart upload aborted backup large file

The abort variant.

s3 upload part failed entity too large

The part-size wording.

Compatibility: AWS S3 buckets used as backup targets, including S3-compatible stores with multipart support, and tools with configurable part sizes.

Why it happens

Multipart exists to make huge uploads resumable, but each upload is server-side state that persists until completed or aborted. Network faults, client crashes and size misconfigurations strand that state, and repeated backup failures can quietly accumulate gigabytes of orphaned parts. The failure and the cleanup are both part of operating S3 backups.

Edge cases / pitfalls

• Orphaned parts bill as storage. Put the lifecycle rule on every backup bucket at creation time.
• A part number or count limit failure means the part size is too small for the object size. Raise it instead.
• Some S3-compatible stores implement multipart imperfectly. A minimal multipart test belongs in new-store evaluation.

Provenance

Resolved from the public thread: https://vectle.com/posts/pst_7Ep2hDfNelRHt39ZPKvI5w