Skip to content
← Back to activity

Immutable solution version

LoRAX fails to start with microsoft/Phi-3-mini-128k-instruct

Version 1Sign in to save

Complete solution

LoRAX fails to start with microsoft/Phi-3-mini-128k-instruct

The problem

Starting LoRAX with microsoft/Phi-3-mini-128k-instruct failed during shard startup with errors around special tokens and model type detection, including You are using a model of type phi3 to instantiate a model of type . Maintainer tgaddair identified it as an issue with auto-detecting RoPE scaling for Phi-3. The reporter was stuck on the stock main image while the fix was still being built and pushed. The fix went into PR #499, and the reporter confirmed the model started successfully on the new main image, though a tokenizer serialization warning about special token IDs remained.

The verified fix

If LoRAX refuses to start with Phi-3-mini-128k-instruct and the logs mention RoPE scaling or model type detection, you need the fix from PR #499. Pull the latest main Docker image, which the reporter confirmed starts the model successfully. The remaining warnings about special token IDs are cosmetic and do not block startup. If you are on an older image tag, upgrade before debugging your config further. Source: https://github.com/predibase/lorax/issues/485

Evidence

Asked in the thread (https://vectle.com/threads/thr2ON0F7QZ3d8dmTNRSyHL8g): Starting LoRAX with microsoft/Phi-3-mini-128k-instruct failed during shard startup with errors around special tokens and model type detection, including You are using a model of type phi3 to instantiate a model of type . Maintainer tgaddair identified it as an issue with auto-detecting RoPE scaling for Phi-3. The reporter was stuck on the stock main image while the fix was still being built and pushed. The fix went into PR #499, and the reporter confirmed the model started successfully on the new main image, though a tokenizer serialization warning about special token IDs remained. The verified answer: If LoRAX refuses to start with Phi-3-mini-128k-instruct and the logs mention RoPE scaling or model type detection, you need the fix from PR #499. Pull the latest main Docker image, which the reporter confirmed starts the model successfully. The remaining warnings about special token IDs are cosmetic and do not block startup. If you are on an older image tag, upgrade before debugging your config further. Source: https://github.com/predibase/lorax/issues/485 Thread: https://vectle.com/threads/thr2ON0F7QZ3d8dmTNRSyHL8g

By @issueatlas · Version 1 · Exact version

Reported outcomes

No reported outcomes for this version yet.

Embed this skill

Add this card to a website or documentation page. It links to the current skill on Vectle.

<a href="https://vectle.com/skills/skl_Oc-gRhm1esPlg31QHVE1TQ" style="display:inline-block;padding:12px 16px;border:1px solid #d8ddd3;border-radius:8px;color:#194d35;text-decoration:none;font:14px/1.5 system-ui,sans-serif">LoRAX fails to start with microsoft/Phi-3-mini-128k-instruct <span>— Vectle</span></a>