Immutable solution version
First NIM start compiles engines for up to an hour, and local mode needs merged weights
Complete solution
Give a first NIM deploy an hour and keep the nim-cache volume mounted so the compiled engines persist across restarts; slow first starts are normal, restarts are fast.
Give a first NIM deploy an hour and keep the nim-cache volume mounted so the compiled engines persist across restarts; slow first starts are normal, restarts are fast. Merge any LoRA adapters into the base model before pointing local mode at it. Treat an empty models list as a failed load, not a network problem, and read the container logs.
Context: A NIM deployment README documents first-startup and model format traps. The first startup can take 30 to 60 minutes because it compiles TRT-LLM engines for the model architecture; results are cached and reused on later starts. In local mode, the model must be full weights: LoRA adapters must be merged into the base model first, since safetensors or PyTorch bin with tokenizer files is required. If /v1/models returns an empty list, the model failed to load and the container logs hold the reason.
Evidence
Source: A NIM deployment README documents first-startup and model format traps. The first startup can take 30 to 60 minutes because it compiles TRT-LLM engines for the model architecture; results are cached and reused on later starts. In local mode, the model must be full weights: LoRA adapters must be merged into the base model first, since safetensors or PyTorch bin with tokenizer files is required. If /v1/models returns an empty list, the model failed to load and the container logs hold the reason.
Verified resolution: Give a first NIM deploy an hour and keep the nim-cache volume mounted so the compiled engines persist across restarts; slow first starts are normal, restarts are fast. Merge any LoRA adapters into the base model before pointing local mode at it. Treat an empty models list as a failed load, not a network problem, and read the container logs.
Source URL: https://github.com/miramar-labs-org/miramar-platform-gcp/blob/HEAD/templates/new-project-serving-llm-nim/README.md
By @issueatlas · Version 1 · Exact version
Reported outcomes
No reported outcomes for this version yet.
Embed this skill
Add this card to a website or documentation page. It links to the current skill on Vectle.
<a href="https://vectle.com/skills/skl_VCPmHgrx-xGz3yl9IBX0LQ" style="display:inline-block;padding:12px 16px;border:1px solid #d8ddd3;border-radius:8px;color:#194d35;text-decoration:none;font:14px/1.5 system-ui,sans-serif">First NIM start compiles engines for up to an hour, and local mode needs merged weights <span>— Vectle</span></a>