Notes | Andy Arditi
Notes This is a rough collection of notes on various technical topics. These notes will always be incomplete, and a work in progress. Additionally, many of these notes are just my reformulation of existing work - see the “Sources” section in each note to see the original source of the material.
This is a rough collection of notes on various technical topics. These notes will always be incomplete, and a work in progress. Additionally, many of these notes are just my reformulation of existing work - see the “Sources” section in each note to see the original source of the material. Multivariate Gaussians Random vectors • Multivariate Gaussian distribution • Disentangling the multivariate Gaussian • References Concentration inequalities Markov’s inequality • Chebyshev’s inequality • Chernoff’s inequality • References Attention Query, key, and value projections • Attention…
saved by
related reading
- Deriving Muonjeremybernste.in
- Muon: An optimizer for hidden layers in neural networks | Keller Jordan blogkellerjordan.github.io
- Modular Manifolds - Thinking Machines Labthinkingmachines.ai
- Some Math behind Neural Tangent Kernel | Lil'Loglilianweng.github.io
- Understanding the Neural Tangent Kernel – EigenTaleseigentales.com
- Understanding Muonlakernewhouse.com
- Gregory Gundersengregorygundersen.com
- [2604.04891] Muon Dynamics as a Spectral Wasserstein Flowarxiv.org
- Neural tangent kernel - Wikipediaen.wikipedia.org
- Why Momentum Really Worksdistill.pub
- microgptkarpathy.github.io
- Understanding Deep Learningudlbook.github.io