Not to able to serve mistralai/Mixtral-8x7B-Instruct-v0.1 on loraX with multiple GPU's
Pull the latest LoRAX image and retry:
docker pull ghcr.io/predibase/lorax:latest
The maintainer suspected a stale image, and the reporter confirmed multi-GPU serving worked after updating. For NVLink setups, also set `NCCL_P2P_LEVEL=NVL`; the reporter confirmed that combination worked too.
_Source: gh:predibase/lorax#191 (Predibase)_