siboehm

Simon Boehm's technical blog

siboehm This is the website of Simon Boehm. If you like these posts: Subscribe to the mailing list. Can Function Inlining Affect Floating Point Outputs? Exploring FMA and Other Consistency Issues At my job, I’m refactoring a 30k LOC codebase that simulates learning in the mammal brain. The emergent behavior of these large brain models is hard to test for, so we opted to take the safe route and preserve bit-equality in the weights of the trained model to guarantee that we are not breaking anything. The main downside of hash-based regression testing during refactoring is that it doesn’t check nu

saved by

• Linda
• Simon Xiang
• Ishaan Panigrahi

related reading

• How to Optimize a CUDA Matmul Kernel for cuBLAS-like Performance: a Worklogsiboehm.com
• Transformer Inference Arithmetic | kipply's blogkipp.ly
• Modern GPU Programming For MLSys — Modern GPU Programming For MLSysmlc.ai
• KernelBench: Can LLMs Write GPU Kernels?scalingintelligence.stanford.edu
• How GPT, Claude, and Gemini are actually trained and served – Reiner Popeyoutube.com
• Making Deep Learning go Brrrr From First Principleshorace.io
• Inside NVIDIA GPUs: Anatomy of high performance matmul kernels - Aleksa Gordićaleksagordic.com
• Machine Learning System Resources | std::bodun::blogbodunhu.com
• Pipeline-Parallelism: Distributed Training via Model Partitioningsiboehm.com
• Outperforming cuBLAS on H100: a Worklogcudaforfun.substack.com
• ⭐️ Fast LLM Inference From Scratchandrewkchan.dev
• Outperforming cuBLAS on H100: a Worklogcudaforfun.substack.com