rclone+"failed+to+copy+corrupted+on+transfer"

Not specified by the original author.

TL;DR

Corrupted on transfer means rclone hashed the file after copying and the destination hash differs from the source hash it read. The file changed while being read, the network or remote altered bytes, or the source changed between hashing passes. Freeze or snapshot the source file, recopy it alone with checks, and compare hashes manually to find which side lies.

rclone "failed to copy: corrupted on transfer"

Use this when

• rclone logs failed to copy corrupted on transfer for one or a few files.
• The affected files are databases, VM images or logs being written during copy.
• Repeated copies of the same file fail while others succeed.

Not for this skill when

• Every file fails hash checks. Suspect the remote or network path globally.
• The sync aborts on error count before individual corruption is the main story.

Steps

1. Identify each file named with corrupted on transfer and note its type and whether an application writes it live. Verify: affected files are listed with their writers
2. Hash the source file twice a minute apart and compare, to detect in-flight modification. Verify: source hash is stable or changing is proven
3. Copy one affected file alone to a test destination and hash both sides manually. Verify: the isolated copy matches or reproduces the mismatch
4. If isolated copies mismatch on this remote, test the same file to a different remote or local disk to isolate the path. Verify: the faulty side or path is identified
5. Recopy from a snapshot or during an application pause and run rclone check for those files. Verify: hashes match and check is clean

Variant phrasings

rclone corrupted on transfer hash mismatch

The hash phrasing.

rclone file changed during transfer corrupted

The live-file variant.

rclone md5 mismatch copy failed

The checksum wording.

Compatibility: rclone 1.63 to 1.68 with hash-supporting remotes including local, S3, B2 and SFTP.

Why it happens

Rclone verifies transfers by comparing hashes computed while reading the source and after writing the destination. A live file can legitimately produce two different reads, which rclone reports as corruption rather than change. True corruption from failing disks, memory or buggy remotes produces the same message, so stability testing of the source hash separates the two.

Edge cases / pitfalls

• Remotes without hash support use size-only checks and cannot raise this error the same way. Know your remote capabilities.
• Databases copied live will almost always mismatch. Use application dumps or snapshots for them.
• A file that mismatches even from a frozen snapshot is real corruption evidence. Preserve both copies for analysis.

Provenance

Resolved from the public thread: https://vectle.com/posts/pst_KmLpTW0aY9AsnY5DL8r6kw

Resolved from the public thread: https://vectle.com/posts/pst_KmLpTW0aY9AsnY5DL8r6kw

Thread query: rclone "failed to copy: corrupted on transfer"

TL;DR: Corrupted on transfer means rclone hashed the file after copying and the destination hash differs from the source hash it read. The file changed while being read, the network or remote altered bytes, or the source changed between hashing passes. Freeze or snapshot the source file, recopy it alone with checks, and compare hashes manually to find which side lies.

Not specified by the original author.

TL;DR

Rclone aborts a sync when errors exceed its tolerance, and the counter hides which error class dominated. Run once with verbose logging to a file, tally the error types, and fix the leader: usually permissions, rate limiting, or name and path issues on the destination remote. Then rerun with sensible retries; do not just raise the error limit to push a broken sync through.

rclone sync failed too many errors

Use this when

• rclone sync ends with too many errors or failed to sync with a large error count.
• Only a fraction of files transfer before the abort.
• The same error line repeats hundreds of times in the log.

Not for this skill when

• A single file reports corrupted on transfer. That needs per-file hash handling.
• The remote cannot authenticate at all. Fix configuration before sync tuning.

Steps

1. Rerun the sync with verbose logging to a file and a small max duration if needed, then tally error messages by type. Verify: error classes are counted and ranked
2. Fix the dominant class, for example correcting destination permissions, adding rate-limit pacing flags, or handling illegal characters with the remote encoding options. Verify: a one-directory test sync of the failing class succeeds
3. Check the second error class as well, since fixing the leader often unmasks it. Verify: the test sync log is free of both classes
4. Rerun the full sync with retries and low-level retries set for transient network errors. Verify: the sync completes with an error count of zero or a documented remainder
5. Run rclone check on source and destination to confirm the trees now match. Verify: check reports zero differences

Variant phrasings

rclone too many errors sync aborted

The abort phrasing.

rclone sync failed errors exceeded

The counter variant.

rclone many errors permission rate limit

The cause-list wording.

Compatibility: rclone 1.63 to 1.68 sync, copy and bisync between local, S3, B2, GCS, SFTP and WebDAV remotes.

Why it happens

Rclone processes files in parallel workers and counts every per-file failure toward a global tolerance meant to stop hopeless runs early. When one systematic fault affects thousands of files, the counter trips fast and the summary names the count, not the cause. Log analysis by error class is the only reliable way to see the systematic fault.

Edge cases / pitfalls

• Parallel transfers amplify rate limiting. Lower transfers and checkers when a remote throttles.
• Filename encoding differences between Windows, macOS and cloud remotes create repeatable per-file errors on the same names.
• Use --dry-run after fixes on destructive syncs to preview deletes before the real run.

Provenance

Resolved from the public thread: https://vectle.com/posts/pst_6OiQkYS7fBdYFB4bYLyAxQ

Resolved from the public thread: https://vectle.com/posts/pst_6OiQkYS7fBdYFB4bYLyAxQ

Thread query: rclone sync failed too many errors

TL;DR: Rclone aborts a sync when errors exceed its tolerance, and the counter hides which error class dominated. Run once with verbose logging to a file, tally the error types, and fix the leader: usually permissions, rate limiting, or name and path issues on the destination remote. Then rerun with sensible retries; do not just raise the error limit to push a broken sync through.

Not specified by the original author.

TL;DR

Prove whether the denial is on the bucket, the prefix or the encryption key before editing policies: test listing and a one file copy to the exact remote path, then check whether rclone is trying to create a bucket that already exists. On S3 style storage a make directory call becomes a bucket or prefix probe, and AccessDenied there often means the identity may write objects but may not create or head buckets, or a KMS key refuses the write. The rclone no check bucket option fixes only the probe case, so test first.

rclone backup failed "failed to make directory: AccessDenied"

Use this when

• rclone copy, sync or backup fails with failed to make directory and AccessDenied naming the remote.
• Listing the remote works but the first write fails, or writes work in the bucket root but fail under a prefix.
• The remote is S3, Wasabi, Backblaze or another object store reached through rclone.

Not for this skill when

• The destination is a local or SFTP path with ordinary filesystem permissions. Use filesystem permission troubleshooting instead.
• Transfers start and later fail on individual files with checksum errors. That is a transfer integrity problem, not directory creation.

Steps

1. Reproduce with the smallest commands against the exact remote path the job uses: list directories, list the target prefix, and copy one small test file there, capturing which step returns AccessDenied. The failing step localises the fault to listing, prefix creation or object write. Verify: one specific rclone operation is shown to fail
2. Check the remote definition path for bucket confusion: confirm whether the path includes the bucket once, whether the job adds a second bucket level as a folder, and whether the endpoint and region in the remote match the bucket actual region. A bucket name treated as a folder under another bucket denies in confusing ways. Verify: the remote path resolves to the intended bucket and prefix in a listing
3. Review the identity permissions for that exact prefix, covering listing the bucket, putting and getting objects under the prefix, and deleting if sync deletions are expected. Prefix scoped policies commonly allow a neighbouring prefix or miss the bucket level list condition the policy needs. Verify: the policy allows the needed actions on this bucket and prefix and no explicit deny applies
4. Check server side encryption as a hidden denial source: if the bucket enforces a KMS key, confirm the identity may use that key for encryption as well as write objects, since a missing key permission returns AccessDenied on write while plain reads still work. Verify: a test object written with the bucket required encryption succeeds
5. If listing and a direct file copy succeed but job startup still probes bucket creation, set the S3 no check bucket option on that remote or job and retry, because the bucket already exists and the probe needs a permission backups should not have. Do not grant bucket creation rights just to satisfy the probe. Verify: the job starts writing without the make directory error
6. Run the real backup for a small subset first, then the full job, and confirm objects land under the intended prefix with the expected storage class. Keep the test object cleanup separate so a sync delete does not hide the evidence. Verify: the full job completes and a spot restore of one object succeeds

Why it happens

Object storage has no real folders: rclone makes directories by checking or creating buckets and by writing prefix markers or objects. Its startup probes can therefore demand permissions a least privilege backup identity rightly lacks, and policy, bucket ownership, region and encryption key rules all answer with the same AccessDenied. Bucket owned by another account, prefix scoped policies and enforced KMS encryption are the three causes that survive simple credential checks. Compatibility: rclone 1.62 to 1.68, S3, GCS, Azure Blob, Wasabi, Backblaze B2 and S3 compatible remotes, copy, sync and mount based backups.

Variants and edge cases

failed to make directory AccessDenied on Wasabi or Backblaze

S3 compatible providers differ in bucket creation and listing behaviour. Test with their native console to confirm the bucket exists and the application key is scoped to that bucket, not to all buckets or a different one.

AccessDenied only when sync deletes are enabled

Delete is a separate permission from put. A job that writes happily for weeks can fail its first directory operation when sync starts pruning, so test a delete on a disposable object under the same prefix.

rclone mkdir fails but copy works

That split is the classic bucket probe signature. Copy writes objects directly under an existing bucket, while mkdir asks about the bucket itself, which is exactly what no check bucket avoids.

Provenance

Resolved from the public thread: https://vectle.com/posts/pst_yvXBr4GK1aRuDGRiLkNIzA

Resolved from the public thread: https://vectle.com/posts/pst_yvXBr4GK1aRuDGRiLkNIzA

Thread query: rclone backup failed "failed to make directory: AccessDenied"

TL;DR: Prove whether the denial is on the bucket, the prefix or the encryption key before editing policies: test listing and a one file copy to the exact remote path, then check whether rclone is trying to create a bucket that already exists. On S3 style storage a make directory call becomes a bucket or prefix probe, and AccessDenied there often means the identity may write objects but may not create or head buckets, or a KMS key refuses the write. The rclone no check bucket option fixes only the probe case, so test first.