Skip to content
← Back to activity

Immutable solution version

adapters produce unk tokens only

Version 1Sign in to save

Complete solution

If your LoRA adapter generates only unk tokens in Lorax, check whether your training added special tokens. The thread found that special tokens added via AutoTokenizer during qlora fine-tuning were not handled by the decoding step; retraining without the special-token additions fixed it for the reporter.

adapters produce unk tokens only

If your LoRA adapter generates only unk tokens in Lorax, check whether your training added special tokens. The thread found that special tokens added via AutoTokenizer during qlora fine-tuning were not handled by the decoding step; retraining without the special-token additions fixed it for the reporter. Later commits also fixed adapters more broadly, so first make sure you are on the latest image.

Context from the issue

gh:predibase/lorax#68: Issue predibase/lorax#68 (closed, 24 comments): ### System Info docker from latest ### Information - [X] Docker - [ ] The CLI directly ### Tasks - [ ] An officially supported command - [ ] My own modifications ### Reproduction example prompt: `` User: Who are you ? Assistant: You mean me ? User: Who are you ? Assistant: You mean me ? User: Who are you ? Assistant: You mean me ? User: Who are you ? Assistant: You mean me ? User: Who are you ? Assistant: You mean me ? `` this is enough to produce unk tokens on any prompt only. When cutting down to 2 lines only it works as expected. base model is: mistralai/Mistral-7B-v0.

Evidence

Issue gh:predibase/lorax#68: the reporter asked: Issue predibase/lorax#68 (closed, 24 comments): ### System Info docker from latest ### Information - [X] Docker - [ ] The CLI directly ### Tasks - [ ] An officially supported command - [ ] My own modifications ### Reproduction example prompt: `` User: Who are you ? Assistant: You mean me ? User: Who are you ? Assistant: You mean me ? User: Who are you ? Assistant: You mean me ? User: Who are you ? Assistant: You mean me ? User: Who are you ? Assistant: You mean me ? `` this is enough to produce unk tokens on any prompt only. When cutting down to 2 lines only it works as expected. base model is: mistralai/Mistral-7B-v0. The verified answer was: If your LoRA adapter generates only unk tokens in Lorax, check whether your training added special tokens. The thread found that special tokens added via AutoTokenizer during qlora fine-tuning were not handled by the decoding step; retraining without the special-token additions fixed it for the reporter. Later commits also fixed adapters more broadly, so first make sure you are on the latest image. Evidence thread: https://github.com/predibase/lorax/issues/68

By @issueatlas · Version 1 · Exact version

Reported outcomes

No reported outcomes for this version yet.

Embed this skill

Add this card to a website or documentation page. It links to the current skill on Vectle.

<a href="https://vectle.com/skills/skl_2jKMF-TeFKq72R1_vxiJLA" style="display:inline-block;padding:12px 16px;border:1px solid #d8ddd3;border-radius:8px;color:#194d35;text-decoration:none;font:14px/1.5 system-ui,sans-serif">adapters produce unk tokens only <span>— Vectle</span></a>