azyResearch/ThunderKittens: Tile primitives for speedy kernels

Tile primitives for speedy kernels

ThunderKittens: Tile primitives for speedy kernels ThunderKittens is a framework to make it easy to write fast deep learning kernels in CUDA. It is built around three key principles: Simplicity. ThunderKittens is stupidly simple to write. Extensibility. ThunderKittens is natively embedded into CUDA, so that if you need more than ThunderKittens can offer, it won’t get in your way of building it yourself. Speed. Kernels written in ThunderKittens should be at least as fast as those written from scratch -- especially because ThunderKittens can do things the “right” way under the hood. We…

saved by

• Devansh Pandey

related reading

• [2410.20399] ThunderKittens: Simple, Fast, and Adorable AI Kernelsarxiv.org
• How to Optimize a CUDA Matmul Kernel for cuBLAS-like Performance: a Worklogsiboehm.com
• GPUs Go Brrr · Hazy Researchhazyresearch.stanford.edu
• GitHub - adam-maj/tiny-gpu: A minimal GPU design in Verilog to learn how GPUs work from the ground up · GitHubgithub.com
• Inside NVIDIA GPUs: Anatomy of high performance matmul kernels - Aleksa Gordićaleksagordic.com
• Tiny TPUtinytpu.com
• AMD GPUs go brrr · Hazy Researchhazyresearch.stanford.edu
• Mixture-of-Kittens: our open-source MoE megakernel for NVL72scursor.com
• ⭐️ Fast LLM Inference From Scratchandrewkchan.dev
• GitHub - wafer-ai/gpu-perf-engineering-resources: A curated resource list for learning AI performance engineering, from GPU fundamentals to production inference.github.com
• We reverse-engineered Flash Attention 4modal.com
• KernelBench: Can LLMs Write GPU Kernels?scalingintelligence.stanford.edu