[2502.16982] Muon is Scalable for LLM Training
arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website. Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them. Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs. arXiv Operational Status Get status notifications via email or slack
[2502.16982] Muon is Scalable for LLM Training Skip to main content arXiv is now an independent nonprofit! Learn more × Search arXiv Press Enter to search · Advanced search --> Computer Science > Machine Learning arXiv:2502.16982 (cs) [Submitted on 24 Feb 2025] Title: Muon is Scalable for LLM Training Authors: Jingyuan Liu , Jianlin Su , Xingcheng Yao , Zhejun Jiang , Guokun Lai , Yulun Du , Yidao Qin , Weixin Xu , Enzhe Lu , Junjie Yan , Yanru Chen , Huabin Zheng , Yibo Liu , Shaowei Liu , Bohong Yin , Weiran He , Han Zhu , Yuzhi Wang , Jianzhou Wang , Mengnan Dong , Zheng Zhang
Explore this link on the map →related reading
- How To Scale Your Modeljax-ml.github.io
- Deriving Muonjeremybernste.in
- LoRA Without Regret - Thinking Machines Labthinkingmachines.ai
- LLM Resourcesforrestbicker.com
- frontier model training methodologies | Alex Wa's Blogdjdumpling.github.io
- On neural scaling and the quanta hypothesisericjmichaud.com
- Muon: An optimizer for hidden layers in neural networks | Keller Jordan blogkellerjordan.github.io
- RL Scaling Laws for LLMs - by Cameron R. Wolfe, Ph.D.cameronrwolfe.substack.com
- Composer2.pdfcursor.com
- Bits, FLOPS, and Watts: A Systems-Level Perspective of Scaling LLMs — Part 1 | by Asheesh Goja | Mediummedium.com
- Demystify Transformers: A Guide to Scaling Laws | by Yu-Cheng Tsai | Sage Ai | Mediummedium.com
- Large Language Models Reading List | Sebastian Raschka, PhDsebastianraschka.com