Scientists should use AI as a tool, not an oracle

How AI hype leads to flawed research that fuels more hype

Who produces AI hype? As we discuss in the AI Snake Oil book, it is not just companies and the media but also AI researchers. For example, a pair of widely-publicized papers in Nature in December 2023 claimed to have discovered over 2.2 million new materials using AI, and robotically synthesized 41 of them. Unfortunately, the claims were quickly debunked: “Most of the [41] materials produced were misidentified, and the rest were already known”. As for the large dataset, examining a sample of 250 compounds showed that it was mostly junk. A core selling point of machine learning is discovery…

related reading

• Sloppy Use of Machine Learning Is Causing a ‘Reproducibility Crisis’ in Science | WIREDwired.com
• Introducing the REFORMS checklist for ML-based sciencenormaltech.ai
• The machines are fine. I'm worried about us.ergosphere.blog
• Could AI slow science?normaltech.ai
• ML is useful for many things, but not for predicting scientific replicabilitynormaltech.ai
• I got fooled by AI-for-science hype—here's what it taught meunderstandingai.org
• Four more things we worked on in 2022normaltech.ai
• How much science is verifiable? Results from replicating ICML 2026 oral paperssai.science
• The More You Automate, the Less You See: Hidden Pitfalls of AI Scientist Systemsarxiv.org
• Can AI automate computational reproducibility?normaltech.ai
• The Need for Verification in AI-Driven Scientific Discoveryalphaxiv.org
• Why an overreliance on AI-driven modelling is bad for sciencenature.com