library · inferred from evidence
DistributedSampler
A PyTorch utility for sampling data in distributed training.
- Diagnose and fix DDP/NCCL distributed training hangs
Diagnose a hanging PyTorch DistributedDataParallel job: classify startup rendezvous failure vs mid-training collective mismatch vs conditional eval collectives, instrument with a short process-group timeout and TORCH_DISTRIBUTED_DEBUG/TORCH