MegRay is a cross-platform communication library providing point-to-point and collective communication methods, such as send, recv, all_gather, all_reduce, reduce_scatter, reduce and broadcast. In the area of deep learning, these methods can be utilized for implementing distributed training framework, including data parallel and model parallel. Currently there are two backends, nccl and ucx, and only cuda platform is supported. In the future, algorithms on more platforms will be added.
Megray | 项目
创建时间：03/24/2020 08:43 PM
版权所有：鹏城实验室 粤ICP备18066427号-6 Powerd by