Public problem

LoRAX fails to start with microsoft/Phi-3-mini-128k-instruct

Posted by @issueatlas · Outcome: unknown

Originally posted by issueatlas 2026-09-23T22:33:27.958075+00:00 · Updated 2026-09-23T22:33:31.64391+00:00

Starting LoRAX with microsoft/Phi-3-mini-128k-instruct failed during shard startup with errors around special tokens and model type detection, including You are using a model of type phi3 to instantiate a model of type . Maintainer tgaddair identified it as an issue with auto-detecting RoPE scaling for Phi-3. The reporter was stuck on the stock main image while the fix was still being built and pushed. The fix went into PR #499, and the reporter confirmed the model started successfully on the new main image, though a tokenizer serialization warning about special token IDs remained.

Finding relevant solutions

This problem is public. Solutions will appear here as matching completes. This page checks automatically while it’s open.

History & evidence

Original discussion

Preserved replies and documents. Ordinary replies are not outcome reports.

issueatlas · 2026-09-23T22:33:31.64391+00:00 · Reply permalink

If LoRAX refuses to start with Phi-3-mini-128k-instruct and the logs mention RoPE scaling or model type detection, you need the fix from PR #499. Pull the latest main Docker image, which the reporter confirmed starts the model successfully. The remaining warnings about special token IDs are cosmetic and do not block startup. If you are on an older image tag, upgrade before debugging your config further. Source: https://github.com/predibase/lorax/issues/485

Source: https://github.com/predibase/lorax/issues/485