PyTorch + NumPy Bug
tanelp.github.io · 1,816 words · saved by 2 readers
A bug that plagues thousands of open-source ML projects.
Update: The problem is now fixed in Pytorch but can still happen in tensorflow-keras. Discussion on reddit . Bugs in ML code are notoriously hard to fix - they don’t cause compile errors but silently regress accuracy. Once you have endured the pain and fixed one of these, the lesson is forever etched into your brain, right? Wrong . Recently, an old foe made a comeback - a familiar bug bit me again! As before, the performance improved significantly after fixing it. The bug was subtle and easy to make. How many others has it done damage to? Curious, I downloaded over a hundred thousand repositor
saved by
related reading
- the bug that taught me more about PyTorch than years of using it | Elana Simonelanapearl.github.io
- Defeating Nondeterminism in LLM Inference - Thinking Machines Labthinkingmachines.ai
- PyTorch internals : ezyang's blogblog.ezyang.com
- GitHub - srush/Tensor-Puzzles: Solve puzzles. Improve your pytorch.github.com
- Tensors — PyTorch Tutorials 2.12.0+cu130 documentationpytorch.org
- PiTorch: ML on Baremetal Raspberry Pis | projectsmasonjwang.com
- Tensors — PyTorch Tutorials 2.13.0+cu130 documentationdocs.pytorch.org
- Open Weights isn't Open Trainingworkshoplabs.ai
- PyTorch in One Hour: From Tensors to Training Neural Networks on Multiple GPUssebastianraschka.com
- GPT in 60 Lines of NumPy | Jay Modyjaykmody.com
- PyTorch (@PyTorch) / Xx.com
- Accelerating Generative AI with PyTorch II: GPT, Fast – PyTorchpytorch.org