Muon Outperforms Adam in Tail-End Associative Memory Learning
arxiv.org · 5,602 words · saved by 1 readers
N/A
Muon Outperforms Adam in Tail-End Associative Memory Learning Shuche Wang1,∗ Fengzhuo Zhang1,∗,† Jiaxiang Li2,∗ Cunxiao Du3 Chao Du3 Tianyu Pang3 Zhuoran Yang4 Mingyi Hong2 Vincent Y. F. Tan1 1 National University of Singapore 2 University of Minnesota 3…
related reading
- Deriving Muonjeremybernste.in
- Muon: An optimizer for hidden layers in neural networks | Keller Jordan blogkellerjordan.github.io
- Does Muon improve regulatory DNA learning? Part 1. — Origin Bioorigin.bio
- LoRA Without Regret - Thinking Machines Labthinkingmachines.ai
- [2502.16982] Muon is Scalable for LLM Trainingarxiv.org
- NL.pdfabehrouz.github.io
- [2505.21829] In Search of Adam's Secret Saucearxiv.org
- Aman's AI Journal • Primers • Ilya Sutskever's Top 30aman.ai
- The Big LLM Architecture Comparisonmagazine.sebastianraschka.com
- [2501.00663] Titans: Learning to Memorize at Test Timearxiv.org
- Understanding Muonlakernewhouse.com
- Adam Optimizer Causes Privileged Basis in Transformer LM Residual Stream — LessWronglesswrong.com