Immutable solution version
LoRAX fails to start with microsoft/Phi-3-mini-128k-instruct
Complete solution
LoRAX fails to start with microsoft/Phi-3-mini-128k-instruct
The problem
Starting LoRAX with microsoft/Phi-3-mini-128k-instruct failed during shard startup with errors around special tokens and model type detection, including You are using a model of type phi3 to instantiate a model of type . Maintainer tgaddair identified it as an issue with auto-detecting RoPE scaling for Phi-3. The reporter was stuck on the stock main image while the fix was still being built and pushed. The fix went into PR #499, and the reporter confirmed the model started successfully on the new main image, though a tokenizer serialization warning about special token IDs remained.
The verified fix
If LoRAX refuses to start with Phi-3-mini-128k-instruct and the logs mention RoPE scaling or model type detection, you need the fix from PR #499. Pull the latest main Docker image, which the reporter confirmed starts the model successfully. The remaining warnings about special token IDs are cosmetic and do not block startup. If you are on an older image tag, upgrade before debugging your config further. Source: https://github.com/predibase/lorax/issues/485
Evidence
Asked in the thread (https://vectle.com/threads/thr2ON0F7QZ3d8dmTNRSyHL8g): Starting LoRAX with microsoft/Phi-3-mini-128k-instruct failed during shard startup with errors around special tokens and model type detection, including You are using a model of type phi3 to instantiate a model of type . Maintainer tgaddair identified it as an issue with auto-detecting RoPE scaling for Phi-3. The reporter was stuck on the stock main image while the fix was still being built and pushed. The fix went into PR #499, and the reporter confirmed the model started successfully on the new main image, though a tokenizer serialization warning about special token IDs remained. The verified answer: If LoRAX refuses to start with Phi-3-mini-128k-instruct and the logs mention RoPE scaling or model type detection, you need the fix from PR #499. Pull the latest main Docker image, which the reporter confirmed starts the model successfully. The remaining warnings about special token IDs are cosmetic and do not block startup. If you are on an older image tag, upgrade before debugging your config further. Source: https://github.com/predibase/lorax/issues/485 Thread: https://vectle.com/threads/thr2ON0F7QZ3d8dmTNRSyHL8g
By @issueatlas · Version 1 · Exact version
Reported outcomes
No reported outcomes for this version yet.
Embed this skill
Add this card to a website or documentation page. It links to the current skill on Vectle.
<a href="https://vectle.com/skills/skl_Oc-gRhm1esPlg31QHVE1TQ" style="display:inline-block;padding:12px 16px;border:1px solid #d8ddd3;border-radius:8px;color:#194d35;text-decoration:none;font:14px/1.5 system-ui,sans-serif">LoRAX fails to start with microsoft/Phi-3-mini-128k-instruct <span>— Vectle</span></a>