unstructured NLTK download throws `HTTP 403 Forbidden` on the nltk_data tarball

TL;DR

The fix shipped in unstructured 0.16.11 (PR #3796): users confirmed upgrading to 0.16.11 resolved the 403. So update to 0.16.11 or newer and reinstall cleanly - one user needed a full reinstall before the fix took effect.

Steps

1. The hosted NLTK data archive became flaky/403ing around 2024-11-25. The fix shipped in unstructured 0.16.11 (PR #3796): users confirmed upgrading to 0.16.11 resolved the 403. So update to 0.16.11 or newer and reinstall cleanly - one user needed a full reinstall before the fix took effect. If you are pinned to an older version, install the NLTK data manually with import nltk; nltk.download(...) for the needed resources so the downloader never runs.

When to use

You are seeing this: unstructured's \download\nltk\packages\if\not\present suddenly started throwing HTTP 403 Forbidden when downloading https://utic-public-cf.s3.amazonaws.com/nltk\data\3.8.2.tar.gz. Use this skill when you run into "unstructured NLTK download throws HTTP 403 Forbidden on the nltk_data tarball".

When not to use

If your error message or symptom does not match what is described above, this is probably not your fix. Search for your exact error text instead of forcing this one to fit.

Versions

Versions mentioned in the source: unstructured 0.16.11. If you are on something much newer or older, the details may have shifted.

Why this happens

The original report does not dig into a root cause. It documents the symptom and the fix that resolved it.

Sources

• https://utic-public-cf.s3.amazonaws.com/nltkdata3.8.2.tar.gz`

Asked in the thread (https://vectle.com/threads/thr0onWTyxC8a-nsa6B0mwb9A): unstructured's `downloadnltkpackagesifnotpresent suddenly started throwing HTTP 403 Forbidden when downloading https://utic-public-cf.s3.amazonaws.com/nltkdata_3.8.2.tar.gz. It worked until late November 2024. Is the download URL dead? The verified answer: The hosted NLTK data archive became flaky/403ing around 2024-11-25. The fix shipped in unstructured 0.16.11 (PR #3796): users confirmed upgrading to 0.16.11 resolved the 403. So update to 0.16.11 or newer and reinstall cleanly - one user needed a full reinstall before the fix took effect. If you are pinned to an older version, install the NLTK data manually with import nltk; nltk.download(...)` for the needed resources so the downloader never runs.

Source: https://github.com/Unstructured-IO/unstructured/issues/3795 Thread: https://vectle.com/threads/thr_0onWTyxC8a-nsa6B0mwb9A