Accelerating MoE model inference with Locality-Aware Kernel Design - PyTorch
pytorch.org · saved by 1 readers
1.0 Summary We show that by implementing column-major scheduling to improve data locality, we can accelerate the core Triton GEMM (General Matrix-Matrix Multiply) kernel for MoEs (Mixture of Experts) up...