[2310.19726] A Path to Simpler Models Starts With Noise

arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website. Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them. Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs. arXiv Operational Status Get status notifications via email or slack

View PDF HTML (experimental) Abstract:The Rashomon set is the set of models that perform approximately equally well on a given dataset, and the Rashomon ratio is the fraction of all models in a given hypothesis space that are in the Rashomon set. Rashomon ratios are often large for tabular datasets in criminal justice, healthcare, lending, education, and in other areas, which has practical implications about whether simpler models can attain the same level of accuracy as more complex models. An open question is why Rashomon ratios often tend to be large. In this work, we propose and study a…

saved by

• Towaki Takikawa

related reading

• [2404.19756] KAN: Kolmogorov-Arnold Networksarxiv.org
• [2405.07987] The Platonic Representation Hypothesisarxiv.org
• [2512.19428] Attention Is Not What You Needarxiv.org
• Hyper-Connectionsarxiv.org
• LLM Post-Training: A Deep Dive into Reasoning Large Language Modelsarxiv.org
• Noisy Data Breaks RLVRddkang.substack.com
• [2509.14786] Pre-training under infinite computearxiv.org
• [2406.02061] Alice in Wonderland: Simple Tasks Showing Complete Reasoning Breakdown in State-Of-the-Art Large Language Modelsarxiv.org
• Training Strategies for Efficient Embodied Reasoningarxiv.org
• Mechanistic Interpretability of Binary and Ternary Transformersarxiv.org
• [2505.05410] Reasoning Models Don't Always Say What They Thinkarxiv.org
• Reasoning Models Reason Well, Until They Don'tarxiv.org