[2502.16982] Muon is Scalable for LLM Training

arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website. Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them. Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs. arXiv Operational Status Get status notifications via email or slack

[2502.16982] Muon is Scalable for LLM Training Skip to main content arXiv is now an independent nonprofit! Learn more × Search arXiv Press Enter to search · Advanced search --> Computer Science > Machine Learning arXiv:2502.16982 (cs) [Submitted on 24 Feb 2025] Title: Muon is Scalable for LLM Training Authors: Jingyuan Liu , Jianlin Su , Xingcheng Yao , Zhejun Jiang , Guokun Lai , Yulun Du , Yidao Qin , Weixin Xu , Enzhe Lu , Junjie Yan , Yanru Chen , Huabin Zheng , Yibo Liu , Shaowei Liu , Bohong Yin , Weiran He , Han Zhu , Yuzhi Wang , Jianzhou Wang , Mengnan Dong , Zheng Zhang

related reading

• [2510.13786] The Art of Scaling Reinforcement Learning Compute for LLMsarxiv.org
• Large Language Diffusion Modelsarxiv.org
• LLM Post-Training: A Deep Dive into Reasoning Large Language Modelsarxiv.org
• [2512.19428] Attention Is Not What You Needarxiv.org
• [2509.14786] Pre-training under infinite computearxiv.org
• [2309.01809] Are Emergent Abilities in Large Language Models just In-Context Learning?arxiv.org
• [2506.09501] Understanding and Mitigating Numerical Sources of Nondeterminism in LLM Inferencearxiv.org
• [2406.11717] Refusal in Language Models Is Mediated by a Single Directionarxiv.org
• [2503.07003] Large Language Models Often Say One Thing and Do Anotherarxiv.org
• [2312.15159] Understanding the Potential of FPGA-Based Spatial Acceleration for Large Language Model Inferencearxiv.org
• [2311.08692] Routing to the Expert: Efficient Reward-guided Ensemble of Large Language Modelsarxiv.org
• [2506.00751] Alignment Revisited: Are Large Language Models Consistent in Stated and Revealed Preferences?arxiv.org