## TL;DR

Run `nltk.download('punkt_tab')` before you tokenize. Since NLTK 3.8.2 (fully in 3.9), `sent_tokenize` and `word_tokenize` look for the new `punkt_tab` resource instead of the old pickled `punkt` data. Downloading only `punkt` is no longer enough on newer NLTK versions.

## The error

```text
nltk punkt_tab resource not found
```

The full traceback usually reads:

```text
LookupError:
**********************************************************************
  Resource punkt_tab not found.
  Please use the NLTK Downloader to obtain the resource.
```

## Steps

### 1. Check your NLTK version

```python
import nltk
print(nltk.__version__)
```

Expected: you see a version number. If it is 3.8.2 or higher, this error is the known `punkt` to `punkt_tab` rename.

### 2. Download the missing resource

```python
import nltk
nltk.download('punkt_tab')
```

Expected: the downloader reports the resource as downloaded (a green progress line in the notebook, or `[nltk_data] Downloading package punkt_tab to ...`).

### 3. If your code also touches older resources, download both

```python
import nltk
nltk.download('punkt')
nltk.download('punkt_tab')
```

Expected: both downloads complete. Some libraries still reference the old `punkt` name, so grabbing both removes the guesswork.

### 4. Make sure NLTK can find the data where it landed

```python
import nltk
print(nltk.data.path)
```

Expected: one of the listed directories contains a `tokenizers/punkt_tab` folder. If the download went to a different directory than NLTK searches, point it at the right one before downloading:

```python
import nltk
nltk.data.path.append('/path/to/your/nltk_data')
nltk.download('punkt_tab', download_dir='/path/to/your/nltk_data')
```

Expected: `sent_tokenize('Hello world. How are you?')` returns `['Hello world.', 'How are you?']` with no LookupError.

### 5. For Docker or CI, bake the data into the image

Do the download at build time so every container starts with the data, with no runtime download and no slow first request:

```dockerfile
RUN python -c "import nltk; nltk.download('punkt_tab', download_dir='/usr/local/share/nltk_data')"
ENV NLTK_DATA=/usr/local/share/nltk_data
```

Expected: the image builds without a LookupError on cold start, and tokenization works on the first request.

## Use this when

- You call `sent_tokenize` or `word_tokenize` and get `LookupError: Resource punkt_tab not found`
- You upgraded NLTK to 3.8.2+ and tokenization that used to work now fails
- A library you depend on (for example a summarization example script) crashes at evaluation time in a fresh environment
- Tokenization fails inside a Docker container or CI job on first run

## Not for this skill when

- The error names a different resource (like `stopwords` or `wordnet`) - download the resource the error names instead
- Tokenization runs but is slow - that is a performance problem, not a missing-data problem
- The failure is a `UnicodeDecodeError` or an encoding issue with your input text

## Variant phrasings

### LookupError: Resource 'punkt_tab' not found
The exact traceback form. The quoted name tells you exactly what to download: the string inside the quotes is the downloader argument.

### punkt_tab missing on NLTK 3.8.2 and later
Same error, phrased around the version cause. Newer NLTK split the tokenizer data into `punkt` plus `punkt_tab`, and downloading only `punkt` is no longer sufficient.

### word_tokenize crashes in a fresh virtualenv
Fresh environments never have NLTK data. The fix is the same download step; there is nothing wrong with the environment itself.

## Why it happens

NLTK ships its trained tokenizer models as separately downloaded data packages, not inside the pip install. In 3.8.2 the maintainers replaced the pickled `punkt` tables with a new tabular format stored under the name `punkt_tab`, and `sent_tokenize` (which `word_tokenize` uses internally) was switched to load `punkt_tab`. Code written against older NLTK only ever downloaded `punkt`, so after an upgrade the lookup fails with a resource name that looks unfamiliar.

## Edge cases

- **Headless server or CI with no write permission to the default data dir:** download to an explicit directory you control and append it to `nltk.data.path`, as in step 4.
- **No internet access at runtime:** pre-download the data on a connected machine, ship the `nltk_data` folder with your deployment, and set `NLTK_DATA` to its path.
- **The downloader says the resource is up to date but the error persists:** your NLTK install and the data may be on mismatched major versions. Reinstall NLTK and re-download in the same environment.
- **Same class of error, different name:** `averaged_perceptron_tagger` became `averaged_perceptron_tagger_eng` and `maxent_ne_chunker` became `maxent_ne_chunker_tab` in the same rename wave. Always download the exact name the error prints.

## Provenance

Resolved from the public thread: https://vectle.com/posts/pst_zQiM1v2OKgjusXOQyET3Yw