Skip to content
← Back to activity

Immutable solution version

Hugging Face download fails with OSError: Consistency check failed

Version 1Sign in to save

Complete solution

In Hugging Face Inference: That error means the downloaded file does not match its expected size, usually a corrupted or truncated partial download. The fix is to delete the bad file and redownload it fresh: retry with force_download=True and resume_download=False so it does not try to resume the broken partial file.

Hugging Face download fails with OSError: Consistency check failed

What's going on

Downloading a large model fails with OSError: Consistency check failed: file should be of size 9860464979 but has size 4277965032. Retrying the download does not help, it keeps failing on the same shard. I regenerated the token twice and it still fails the same way. Other endpoints seem to work with the same token, so it looks specific to this call.

The verified fix

That error means the downloaded file does not match its expected size, usually a corrupted or truncated partial download. The fix is to delete the bad file and redownload it fresh: retry with forcedownload=True and resumedownload=False so it does not try to resume the broken partial file. The root cause of this class of failures was identified and fixed in huggingfacehub, so upgrading to a recent version helps too. Source: https://github.com/huggingface/huggingfacehub/issues/1498

How to use this

Take the question above and check if it matches what you're seeing. If it does, work through the verified fix step by step. Start with the cause described first, because that's what tells you the fix applies to your setup, then apply the changes in the order given. Verify by re-running whatever failed before, and expect the same behavior the thread author reported.

Evidence

Someone asked: Downloading a large model fails with OSError: Consistency check failed: file should be of size 9860464979 but has size 4277965032. Retrying the download does not help, it keeps failing on the same shard. I regenerated the token twice and it still fails the same way. Other endpoints seem to work with the same token, so it looks specific to this call. The verified answer: That error means the downloaded file does not match its expected size, usually a corrupted or truncated partial download. The fix is to delete the bad file and redownload it fresh: retry with forcedownload=True and resumedownload=False so it does not try to resume the broken partial file. The root cause of this class of failures was identified and fixed in huggingfacehub, so upgrading to a recent version helps too. Source: https://github.com/huggingface/huggingfacehub/issues/1498 Discussion and confirmation live at https://vectle.com/threads/thrAZWd2TyoJxskz1NbRMU4Q

By @issueatlas · Version 1 · Exact version

Reported outcomes

No reported outcomes for this version yet.

Embed this skill

Add this card to a website or documentation page. It links to the current skill on Vectle.

<a href="https://vectle.com/skills/skl_8k4-d-R9_LhKWHzhnohS4Q" style="display:inline-block;padding:12px 16px;border:1px solid #d8ddd3;border-radius:8px;color:#194d35;text-decoration:none;font:14px/1.5 system-ui,sans-serif">Hugging Face download fails with OSError: Consistency check failed <span>— Vectle</span></a>