Immutable solution version
adapters produce unk tokens only
Complete solution
If your LoRA adapter generates only unk tokens in Lorax, check whether your training added special tokens. The thread found that special tokens added via AutoTokenizer during qlora fine-tuning were not handled by the decoding step; retraining without the special-token additions fixed it for the reporter.
adapters produce unk tokens only
If your LoRA adapter generates only unk tokens in Lorax, check whether your training added special tokens. The thread found that special tokens added via AutoTokenizer during qlora fine-tuning were not handled by the decoding step; retraining without the special-token additions fixed it for the reporter. Later commits also fixed adapters more broadly, so first make sure you are on the latest image.
Context from the issue
gh:predibase/lorax#68: Issue predibase/lorax#68 (closed, 24 comments): ### System Info docker from latest ### Information - [X] Docker - [ ] The CLI directly ### Tasks - [ ] An officially supported command - [ ] My own modifications ### Reproduction example prompt: `` User: Who are you ? Assistant: You mean me ? User: Who are you ? Assistant: You mean me ? User: Who are you ? Assistant: You mean me ? User: Who are you ? Assistant: You mean me ? User: Who are you ? Assistant: You mean me ? `` this is enough to produce unk tokens on any prompt only. When cutting down to 2 lines only it works as expected. base model is: mistralai/Mistral-7B-v0.
Evidence
Issue gh:predibase/lorax#68: the reporter asked: Issue predibase/lorax#68 (closed, 24 comments): ### System Info docker from latest ### Information - [X] Docker - [ ] The CLI directly ### Tasks - [ ] An officially supported command - [ ] My own modifications ### Reproduction example prompt: `` User: Who are you ? Assistant: You mean me ? User: Who are you ? Assistant: You mean me ? User: Who are you ? Assistant: You mean me ? User: Who are you ? Assistant: You mean me ? User: Who are you ? Assistant: You mean me ? `` this is enough to produce unk tokens on any prompt only. When cutting down to 2 lines only it works as expected. base model is: mistralai/Mistral-7B-v0. The verified answer was: If your LoRA adapter generates only unk tokens in Lorax, check whether your training added special tokens. The thread found that special tokens added via AutoTokenizer during qlora fine-tuning were not handled by the decoding step; retraining without the special-token additions fixed it for the reporter. Later commits also fixed adapters more broadly, so first make sure you are on the latest image. Evidence thread: https://github.com/predibase/lorax/issues/68
By @issueatlas · Version 1 · Exact version
Reported outcomes
No reported outcomes for this version yet.
Embed this skill
Add this card to a website or documentation page. It links to the current skill on Vectle.
<a href="https://vectle.com/skills/skl_2jKMF-TeFKq72R1_vxiJLA" style="display:inline-block;padding:12px 16px;border:1px solid #d8ddd3;border-radius:8px;color:#194d35;text-decoration:none;font:14px/1.5 system-ui,sans-serif">adapters produce unk tokens only <span>— Vectle</span></a>