Scientists should use AI as a tool, not an oracle

How AI hype leads to flawed research that fuels more hype

Who produces AI hype? As we discuss in the AI Snake Oil book, it is not just companies and the media but also AI researchers. For example, a pair of widely-publicized papers in Nature in December 2023 claimed to have discovered over 2.2 million new materials using AI, and robotically synthesized 41 of them. Unfortunately, the claims were quickly debunked: “Most of the [41] materials produced were misidentified, and the rest were already known”. As for the large dataset, examining a sample of 250 compounds showed that it was mostly junk. A core selling point of machine learning is discovery…

related reading

• The machines are fine. I'm worried about us.ergosphere.blog
• Designing AI for Disruptive Scienceasimov.press
• Sloppy Use of Machine Learning Is Causing a ‘Reproducibility Crisis’ in Science | WIREDwired.com
• The More You Automate, the Less You See: Hidden Pitfalls of AI Scientist Systemsarxiv.org
• Introducing the REFORMS checklist for ML-based sciencenormaltech.ai
• ML is useful for many things, but not for predicting scientific replicabilitynormaltech.ai
• The Need for Verification in AI-Driven Scientific Discoveryalphaxiv.org
• Self-Fulfilling Misalignment Data Might Be Poisoning Our AI Modelsturntrout.com
• Could AI slow science?normaltech.ai
• How Big Is Our Blindspot? Why AI for Science Needs More Than Better Modelsrenaissancephilanthropy.substack.com
• Eighteen pitfalls to beware of in AI journalismnormaltech.ai
• Why are AI agents lying, cheating and coordinating?yoshuabengio.org