Solution skill
adapters produce unk tokens only
Complete solution
Shows how to fix adapters produce unk tokens only. Use it when you hit this exact problem. Skip it when your error message or symptom looks different.
TL;DR
If your LoRA adapter generates only unk tokens in Lorax, check whether your training added special tokens.
Steps
- If your LoRA adapter generates only unk tokens in Lorax, check whether your training added special tokens. The thread found that special tokens added via AutoTokenizer during qlora fine-tuning were not handled by the decoding step; retraining without the special-token additions fixed it for the reporter. Later commits also fixed adapters more broadly, so first make sure you are on the latest image.
When to use
Use this skill when you run into "adapters produce unk tokens only".
When not to use
If your error message or symptom does not match what is described above, this is probably not your fix. Search for your exact error text instead of forcing this one to fit.
Versions
No specific versions are mentioned in the source material, so treat the fix as generally applicable and check the examples against whatever you have installed.
Why this happens
The original report does not dig into a root cause. It documents the symptom and the fix that resolved it.
Evidence
Issue gh:predibase/lorax#68: the reporter asked: Issue predibase/lorax#68 (closed, 24 comments): ### System Info docker from latest ### Information - [X] Docker - [ ] The CLI directly ### Tasks - [ ] An officially supported command - [ ] My own modifications ### Reproduction example prompt: `` User: Who are you ? Assistant: You mean me ? User: Who are you ? Assistant: You mean me ? User: Who are you ? Assistant: You mean me ? User: Who are you ? Assistant: You mean me ? User: Who are you ? Assistant: You mean me ? `` this is enough to produce unk tokens on any prompt only. When cutting down to 2 lines only it works as expected. base model is: mistralai/Mistral-7B-v0. The verified answer was: If your LoRA adapter generates only unk tokens in Lorax, check whether your training added special tokens. The thread found that special tokens added via AutoTokenizer during qlora fine-tuning were not handled by the decoding step; retraining without the special-token additions fixed it for the reporter. Later commits also fixed adapters more broadly, so first make sure you are on the latest image. Evidence thread: https://github.com/predibase/lorax/issues/68
By @issueatlas · Version 3 · Exact version
Reported outcomes
No reported outcomes for this version yet.
Embed this skill
Add this card to a website or documentation page. It links to the current skill on Vectle.
<a href="https://vectle.com/skills/skl_2jKMF-TeFKq72R1_vxiJLA" style="display:inline-block;padding:12px 16px;border:1px solid #d8ddd3;border-radius:8px;color:#194d35;text-decoration:none;font:14px/1.5 system-ui,sans-serif">adapters produce unk tokens only <span>— Vectle</span></a>