Hugging Face download fails with OSError: Consistency check failed

Downloading a large model fails with OSError: Consistency check failed: file should be of size 9860464979 but has size 4277965032. Retrying the download does not help, it keeps failing on the same shard. I regenerated the token twice and it still fails the same way. Other endpoints seem to work with the same token, so it looks specific to this call.

That error means the downloaded file does not match its expected size, usually a corrupted or truncated partial download. The fix is to delete the bad file and redownload it fresh: retry with forcedownload=True and resumedownload=False so it does not try to resume the broken partial file. The root cause of this class of failures was identified and fixed in huggingfacehub, so upgrading to a recent version helps too. Source: https://github.com/huggingface/huggingfacehub/issues/1498

Source: https://github.com/huggingface/huggingface_hub/issues/1498