Questionable practices in machine learning

This is experimental HTML to improve accessibility. We invite you to report rendering errors. Use Alt+Y to toggle on accessible reporting links and Alt+Shift+Y to toggle off. Learn more about this project and help improve conversions. Evaluating modern ML models is hard. The strong incentive for researchers and companies to report a state-of-the-art result on some metric often leads to questionable research practices (QRPs): bad practices which fall short of outright research fraud. We describe 44 such practices which can undermine reported results, giving examples where possible. Our list emphasises the evaluation of large language models (LLMs) on public benchmarks. We also discuss “irreproducible research practices”, i.e. decisions that make it difficult or impossible for other researchers to reproduce, build on or audit previous research. Keywords: machine learning methodology, contamination, cherrypicking, misreporting If, like truth, falsehood had but one face, we’d be better of

Questionable practices in machine learning \name Gavin Leech \email gavin@arbresearch.com \addr Arb Research \AND \name Juan J Vazquez \email juan@arbresearch.com \addr Arb Research \AND \name Niclas Kupper \addr University of Bath \AND \name Misha Yagudin \addr Arb Research \AND \name Laurence Aitchison \addr University of Bristol Abstract Evaluating modern ML models is hard. The strong incentive for researchers and companies to report a state-of-the-art result on some metric often leads to questionable research practices (QRPs): bad practices which fall short of outright research fraud. We d

related reading

• Reward Hacking in Reinforcement Learning | Lil'Loglilianweng.github.io
• Training a Misaligned Reward Seekeralignment.anthropic.com
• [2604.22082] Removing Sandbagging in LLMs by Training with Weak Supervisionarxiv.org
• Surfacing Pathological Behaviors in Language Models | Transluce AItransluce.org
• Pitfalls in Evaluating Language Model Forecastersarxiv.org
• [2604.16812] Introspection Adapters: Training LLMs to Report Their Learned Behaviorsarxiv.org
• Simulated Users & Sad LLMs1a3orn.com
• [2312.06942] AI Control: Improving Safety Despite Intentional Subversionarxiv.org
• Alignment Faking Mitigationsalignment.anthropic.com
• Auditing language models for hidden objectives — LessWronglesswrong.com
• [2502.05209] Model Tampering Attacks Enable More Rigorous Evaluations of LLM Capabilitiesarxiv.org
• Alignment faking in large language modelsarxiv.org