✳flâneur — a map of the web's best reading

Crafting Efficient Kernels with Epilogue Fusion

blog.fal.ai · 2,571 words · saved by 1 readers

In many ML workloads, a GEMM is followed by small operations like bias, activation, scaling, or type conversion. These ops are cheap in math, but they often cost extra global memory traffic (store GEMM result, read it back, write again). Epilogue fusion is a way to avoid this, we can

In many ML workloads, a GEMM is followed by small operations like bias, activation, scaling, or type conversion. These ops are cheap in math, but they often cost extra global memory traffic (store GEMM result, read it back, write again). Epilogue fusion is a way to avoid this, we can apply these extra ops while the GEMM result is still in registers, right before the final store to global memory. On Hopper and Blackwell, there is also more room to overlap Tensor Core work with other instructions, so doing some extra compute in the epilogue can be even more attractive. Epilogue fusion eliminates

Explore this link on the map →

saved by

related reading