Han Guo on X: "LLM training is built on fast MatMuls. But many surrounding ops still run as memory-bound kernels. CODA reparameterizes them to hide in the matmul’s shadow, fused into its epilogue before results leave the chip. Bonus: LLMs can write fast CODA kernels too (approaching SoLs). https://t.co/cOTeMUr4py" / X
To view keyboard shortcuts, press question mark View keyboard shortcuts Home Explore Notifications Chat Grok Premium Money Bookmarks Creator Studio Articles Profile More Post christina @luoluo Post See new posts Conversation Han Guo @HanGuo97 LLM training is built on fast MatMuls. But many surrounding ops still run as memory-bound kernels. CODA reparameterizes them to hide in the matmul’s shadow, fused into its epilogue before results leave the chip. Bonus: LLMs can write fast CODA kernels too (approaching SoLs). 3:25 PM · May 21, 2026 · 146.8K Views 11 106 555 437 Relevant View quotes Post your reply Reply Han Guo @HanGuo97 · May 21 LLM training is dominated by compute-heavy ops like MatMuls and attention. But it also has many memory-heavy ops: norms, activations, residuals, reductions. These mostly move tensors around. As FP8/NVFP4 make FLOPs cheaper, data movement gets harder to ignore. Fig: ~1B LLaMA-3 Show more 2 3 51 5.7K Han Guo @HanGuo97 · May 21 ML frameworks make trai
Han Guo @HanGuo97 LLM training is built on fast MatMuls. But many surrounding ops still run as memory-bound kernels. CODA reparameterizes them to hide in the matmul’s shadow, fused into its epilogue before results leave the chip. Bonus: LLMs can write fast CODA kernels too (approaching SoLs). 10:25 PM · May 21, 2026 202K Views 16 0 1 6 103 0 1 0 3 689 0 6 8 9 534 0 5 3 4 Read 16 replies
Explore this link on the map →related reading
- How to Land a Frontier Lab Jobvladfeinberg.com
- Look Ma, No Bubbles! Designing a Low-Latency Megakernel for Llama-1B · Hazy Researchhazyresearch.stanford.edu
- How to Optimize a CUDA Matmul Kernel for cuBLAS-like Performance: a Worklogsiboehm.com
- LoRA Without Regret - Thinking Machines Labthinkingmachines.ai
- ⭐️ Fast LLM Inference From Scratchandrewkchan.dev
- Reiner Pope – The math behind how LLMs are trained and serveddwarkesh.com
- Inside NVIDIA GPUs: Anatomy of high performance matmul kernels - Aleksa Gordićaleksagordic.com
- How is LLaMa.cpp possible?finbarr.ca
- Transformers Inference Optimization Toolset | AstraBlogastralord.github.io
- Kevin-32B: Multi-Turn RL for Writing CUDA Kernels | Cognitioncognition.ai
- Composer2.pdfcursor.com
- 1.5x faster MoE training with custom MXFP8 kernels · Cursorcursor.com