Hugging Face download fails with OSError: Consistency check failed

TL;DR

The fix is to delete the bad file and redownload it fresh: retry with forcedownload=True and resumedownload=False so it does not try to resume the broken partial file. The root cause of this class of failures was identified and fixed in huggingface_hub, so upgrading to a recent version helps too.

Steps

1. That error means the downloaded file does not match its expected size, usually a corrupted or truncated partial download. The fix is to delete the bad file and redownload it fresh: retry with forcedownload=True and resumedownload=False so it does not try to resume the broken partial file. The root cause of this class of failures was identified and fixed in huggingfacehub, so upgrading to a recent version helps too. Source: https://github.com/huggingface/huggingfacehub/issues/1498
2. Take the question above and check if it matches what you're seeing. If it does, work through the verified fix step by step. Start with the cause described first, because that's what tells you the fix applies to your setup, then apply the changes in the order given. Verify by re-running whatever failed before, and expect the same behavior the thread author reported.

When to use

You are seeing this: Downloading a large model fails with OSError: Consistency check failed: file should be of size 9860464979 but has size 4277965032. Use this skill when you run into "Hugging Face download fails with OSError: Consistency check failed".

When not to use

If your error message or symptom does not match what is described above, this is probably not your fix. Search for your exact error text instead of forcing this one to fit.

Versions

No specific versions are mentioned in the source material, so treat the fix as generally applicable and check the examples against whatever you have installed.

Why this happens

The original report does not dig into a root cause. It documents the symptom and the fix that resolved it.

Sources

• https://github.com/huggingface/huggingface_hub/issues/1498

Someone asked: Downloading a large model fails with OSError: Consistency check failed: file should be of size 9860464979 but has size 4277965032. Retrying the download does not help, it keeps failing on the same shard. I regenerated the token twice and it still fails the same way. Other endpoints seem to work with the same token, so it looks specific to this call. The verified answer: That error means the downloaded file does not match its expected size, usually a corrupted or truncated partial download. The fix is to delete the bad file and redownload it fresh: retry with forcedownload=True and resumedownload=False so it does not try to resume the broken partial file. The root cause of this class of failures was identified and fixed in huggingfacehub, so upgrading to a recent version helps too. Source: https://github.com/huggingface/huggingfacehub/issues/1498 Discussion and confirmation live at https://vectle.com/threads/thrAZWd2TyoJxskz1NbRMU4Q