[2506.09501] Give Me FP32 or Give Me Death? Challenges and Solutions for Reproducible Reasoning

arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website. Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them. Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs. arXiv Operational Status Get status notifications via email or slack

[2506.09501] Understanding and Mitigating Numerical Sources of Nondeterminism in LLM Inference Skip to main content arXiv is now an independent nonprofit! Learn more × Search arXiv Press Enter to search · Advanced search --> Computer Science > Computation and Language arXiv:2506.09501 (cs) [Submitted on 11 Jun 2025 ( v1 ), last revised 24 Oct 2025 (this version, v2)] Title: Understanding and Mitigating Numerical Sources of Nondeterminism in LLM Inference Authors: Jiayi Yuan , Hao Li , Xinheng Ding , Wenya Xie , Yu-Jhe Li , Wentian Zhao , Kun Wan , Jing Shi , Xia Hu , Zirui Liu Vie

related reading

• Defeating Nondeterminism in LLM Inference - Thinking Machines Labthinkingmachines.ai
• [2506.09501] Understanding and Mitigating Numerical Sources of Nondeterminism in LLM Inferencearxiv.org
• Non-Determinism of “Deterministic” LLM Settingsarxiv.org
• As Rocks May Think | Eric Jangevjang.com
• Prompt Repetition Improves Non-Reasoning LLMsarxiv.org
• Productizing Large Language Modelsblog.replit.com
• Defeating the Training-Inference Mismatch via FP16arxiv.org
• What We’ve Learned From A Year of Building with LLMs – Applied LLMsapplied-llms.org
• State of RL for reasoning LLMs | A. Weersaweers.de
• Non-determinism in GPT-4 is caused by Sparse MoE - 152334H152334h.github.io
• 2410.05229arxiv.org
• Is AI Reasoning Right for the Wrong Reasons? | Quanta Magazinequantamagazine.org