product · inferred from evidence
Mixtral
A product associated with a tensor-parallel head, mentioned in the context of a LoRAX bug.
- LoRAX: TensorParallelHead object has no attribute base_layer when serving Mixtral with an adapter
Starting LoRAX with `mistralai/Mixtral-8x7B-Instruct-v0.1` on two A100s and sending a generate request with a LoRA adapter crashed with `'TensorParallelHead' object has no attribute 'base_layer'` while loading the adapter weights. Maintaine