Sloppy Use of Machine Learning Is Causing a ‘Reproducibility Crisis’ in Science | WIRED

AI hype has researchers in fields from medicine to political science rushing to use techniques that they don’t always understand—causing a wave of spurious results.

Save this story Save this story The AI Database → Application Prediction End User Research Sector Health care Public safety Research Technology Machine learning History shows civil wars to be among the messiest, most horrifying of human affairs. So Princeton professor Arvind Narayanan and his PhD student Sayash Kapoor got suspicious last year when they discovered a strand of political science research claiming to predict when a civil war will break out with more than 90 percent accuracy, thanks to artificial intelligence . A series of papers described astonishing results from using machine lea

related reading

• Scientists should use AI as a tool, not an oraclenormaltech.ai
• Introducing the REFORMS checklist for ML-based sciencenormaltech.ai
• ML is useful for many things, but not for predicting scientific replicabilitynormaltech.ai
• Four more things we worked on in 2022normaltech.ai
• The machines are fine. I'm worried about us.ergosphere.blog
• Why an overreliance on AI-driven modelling is bad for sciencenature.com
• I got fooled by AI-for-science hype—here's what it taught meunderstandingai.org
• Could AI slow science?normaltech.ai
• Too many AI researchers think real-world problems are not relevant | MIT Technology Reviewtechnologyreview.com
• Why are Machine Learning Projects so Hard to Manage? | by Lukas Biewald | Mediummedium.com
• [2108.02497] How to avoid machine learning pitfalls: a guide for academic researchersarxiv.org
• Reproducibility and transparency: what’s going on and how can we helpnature.com