NCCL
It is a library of routines for high-performance communication between multiple GPUs, used in deep learning and high-performance computing (HPC).
It is a library of routines for high-performance communication between multiple GPUs, used in deep learning and high-performance computing (HPC).