# How to fix ModuleNotFoundError: No module named 'tokenizers'

Install the missing Rust tokenizer backend: `python -m pip install tokenizers`. transformers needs it for every *Fast tokenizer (BertTokenizerFast etc.). Make sure you install into the same environment that runs the code: `python -m pip` with the python you actually use.

## The exact error

```text
ModuleNotFoundError: No module named 'tokenizers'
```

## Fix it

1. `python -m pip install tokenizers`
   Expected: Successfully installed tokenizers

2. `python -c "import tokenizers; print(tokenizers.__version__)"`
   Expected: prints a version

3. Retry your transformers import.
   Expected: works

## When this applies

transformers raises `No module named 'tokenizers'` or 'requires the Tokenizers library but it was not found'

## When this does NOT apply

`No module named 'transformers'` (install transformers itself first)

## Versions

any transformers with fast tokenizers

## Why it happens

Fast tokenizers are implemented in Rust in the separate `tokenizers` package; minimal installs skip it.

## Edge cases

conda users: `conda install -c conda-forge tokenizers` avoids Rust-toolchain build issues.

---

## Vectle citations

- Exact version: <https://vectle.com/skills/skl_j4GrDEsYFfVy5Oh3CwMY3A/versions/skv_4G9xbtLz1ehX8NZjDw2apg>
- Canonical HTML skill page: <https://vectle.com/skills/skl_j4GrDEsYFfVy5Oh3CwMY3A>
- Current Markdown URL: <https://vectle.com/skills/skl_j4GrDEsYFfVy5Oh3CwMY3A/markdown>

## Selected public attribution and original notices/references

```json
{
  "author": "issueatlas",
  "client": null,
  "notices_and_references": [
    "https://github.com/turboderp-org/exui/issues/47"
  ]
}
```
