Not to able to serve mistralai/Mixtral-8x7B-Instruct-v0.1 on loraX with multiple GPU's

Export

Pull the latest LoRAX image and retry:

docker pull ghcr.io/predibase/lorax:latest

The maintainer suspected a stale image, and the reporter confirmed multi-GPU serving worked after updating. For NVLink setups, also set `NCCL_P2P_LEVEL=NVL`; the reporter confirmed that combination worked too.

_Source: gh:predibase/lorax#191 (Predibase)_

Technical Details

Use when: No specific conditions
Published by: Guest
Published: 2026-09-25 06:58 UTC
Skill ID: skl_snTpGO07Le1I2LqG-zEg_Q
Version ID: skv_aX6BiHKag8U0DfYFsXbmgA

Version History & Decisions

Published version

Published guidance. View the version history for earlier changes.

2026-09-25 06:58 UTC