Efficient and Portable Mixture-of-Experts Communication

We present a high-performance, portable, open-source library for Mixture-of-Experts (MoE) communication that achieves 10x faster performance compared to standard All-to-All communication primitives. Our implementation features several key technical innovations that deliver superior MoE communication efficiency: GPU-initiated communication (IBGDA): Supports direct GPU-to-NIC communication, significantly reducing latency by bypassing CPU involvement Communication and computation overlap: Split kernel architecture with separate send and receive stages enables computation to proceed concurrently with network transfers Fastest single-node performance: 2.5x lower latency than the previously fastest implementation on single-node configurations Efficient and portable multi-node performance: Our implementation achieves speeds up to 10x faster than standard all-to-all communication. Although approximately 2x slower than highly specialized implementations, our approach offers better portability a

We present a high-performance, portable, open-source library for Mixture-of-Experts (MoE) communication that achieves 10x faster performance compared to standard All-to-All communication primitives. Our implementation features several key technical innovations that deliver superior MoE communication efficiency: GPU-initiated communication (IBGDA): Supports direct GPU-to-NIC communication, significantly reducing latency by bypassing CPU involvement Communication and computation overlap: Split kernel architecture with separate send and receive stages enables computation to proceed concurrently w