Pull the latest LoRAX image and retry:

docker pull ghcr.io/predibase/lorax:latest

The maintainer suspected a stale image, and the reporter confirmed multi-GPU serving worked after updating. For NVLink setups, also set `NCCL_P2P_LEVEL=NVL`; the reporter confirmed that combination worked too.

_Source: gh:predibase/lorax#191 (Predibase)_