Han Guo on X: "LLM training is built on fast MatMuls. But many surrounding ops still run as memory-bound kernels. CODA reparameterizes them to hide in the matmul’s shadow, fused into its epilogue before results leave the chip. Bonus: LLMs can write fast CODA kernels too (approaching SoLs). https://t.co/cOTeMUr4py" / X

To view keyboard shortcuts, press question mark View keyboard shortcuts Home Explore Notifications Chat Grok Premium Money Bookmarks Creator Studio Articles Profile More Post christina @luoluo Post See new posts Conversation Han Guo @HanGuo97 LLM training is built on fast MatMuls. But many surrounding ops still run as memory-bound kernels. CODA reparameterizes them to hide in the matmul’s shadow, fused into its epilogue before results leave the chip. Bonus: LLMs can write fast CODA kernels too (approaching SoLs). 3:25 PM · May 21, 2026 · 146.8K Views 11 106 555 437 Relevant View quotes Post your reply Reply Han Guo @HanGuo97 · May 21 LLM training is dominated by compute-heavy ops like MatMuls and attention. But it also has many memory-heavy ops: norms, activations, residuals, reductions. These mostly move tensors around. As FP8/NVFP4 make FLOPs cheaper, data movement gets harder to ignore. Fig: ~1B LLaMA-3 Show more 2 3 51 5.7K Han Guo @HanGuo97 · May 21 ML frameworks make trai

Han Guo @HanGuo97 LLM training is built on fast MatMuls. But many surrounding ops still run as memory-bound kernels. CODA reparameterizes them to hide in the matmul’s shadow, fused into its epilogue before results leave the chip. Bonus: LLMs can write fast CODA kernels too (approaching SoLs). 10:25 PM · May 21, 2026 202K Views 16 0 1 6 103 0 1 0 3 689 0 6 8 9 534 0 5 3 4 Read 16 replies

related reading

• CODA: Rewriting Transformer Blocks as GEMM-Epilogue Programsarxiv.org
• ⭐️ Fast LLM Inference From Scratchandrewkchan.dev
• How to Optimize a CUDA Matmul Kernel for cuBLAS-like Performance: a Worklogsiboehm.com
• Inside NVIDIA GPUs: Anatomy of high performance matmul kernels - Aleksa Gordićaleksagordic.com
• Look Ma, No Bubbles! Designing a Low-Latency Megakernel for Llama-1B · Hazy Researchhazyresearch.stanford.edu
• Can LLMs Be Computers?percepta.ai
• Horace He: Building Machine Learning Systems for a Trillion Trillion Floating Point Operationsyoutube.com
• Paged Attention from First Principles: A View Inside vLLM – Hamza's Bloghamzaelshafie.bearblog.dev
• Meta harness makes 10 times better Kimi K3 chipluoluo.ai
• How is LLaMa.cpp possible?finbarr.ca
• Transformers Inference Optimization Toolset | AstraBlogastralord.github.io
• 1.5x faster MoE training with custom MXFP8 kernels · Cursorcursor.com