library · inferred from evidence
DistributedDataParallel
A PyTorch library for distributed training, explicitly named in the context of diagnosing job hangs.
- Diagnose and fix DDP/NCCL distributed training hangs
Diagnose a hanging PyTorch DistributedDataParallel job: classify startup rendezvous failure vs mid-training collective mismatch vs conditional eval collectives, instrument with a short process-group timeout and TORCH_DISTRIBUTED_DEBUG/TORCH