Muon Outperforms Adam in Tail-End Associative Memory Learning
arxiv.org · 5,602 words · saved by 1 readers
N/A
Muon Outperforms Adam in Tail-End Associative Memory Learning Shuche Wang1,∗ Fengzhuo Zhang1,∗,† Jiaxiang Li2,∗ Cunxiao Du3 Chao Du3 Tianyu Pang3 Zhuoran Yang4 Mingyi Hong2 Vincent Y. F. Tan1 1 National University of Singapore 2 University of Minnesota 3…
related reading
- Deriving Muonjeremybernste.in
- Muon: An optimizer for hidden layers in neural networks | Keller Jordan blogkellerjordan.github.io
- transformer_attention.pdfarxiv.org
- Does Muon improve regulatory DNA learning? Part 1. — Origin Bioorigin.bio
- [2501.00663] Titans: Learning to Memorize at Test Timearxiv.org
- [2601.08393] Controlled LLM Training on Spectral Spherearxiv.org
- 2502.11089arxiv.org
- [2411.19870] DeMo: Decoupled Momentum Optimizationarxiv.org
- Understanding Muonlakernewhouse.com
- Scalable MatMul-free Language Modelingarxiv.org
- [2605.15156] MeMo: Memory as a Modelarxiv.org
- 2403.09611.pdfarxiv.org