Why advanced machine learning methods badly overfit niche models – is this statistical machismo? | Dynamic Ecology

Statistical machismo, as I’ve been defining it in this blog (here and here) is when there is a push to ever newer, more complex methods, often in pursuit of a one-dimensional form of improvem…

Statistical machismo, as I’ve been defining it in this blog ( here and here ) is when there is a push to ever newer, more complex methods, often in pursuit of a one-dimensional form of improvement while ignoring the fact that choosing the proper statistic involves complex trade-offs. For example, hyperfocusing on one source of error while ignoring (or even worsening) other sources of error. Here I want to talk about how the new and the complex sides of statistical machismo interact badly in the most common application of machine learning to ecology to date: niche models (aka species dist

related reading

• Perspectives in machine learning for wildlife conservationnature.com
• Scaling Laws, Carefully | Lil'Loglilianweng.github.io
• Too much efficiency makes everything worse: overfitting and the strong version of Goodhart’s law | Jascha’s blogsohl-dickstein.github.io
• Scaling is subtler than it seemsberen.io
• Orthogonal/Double Machine Learning - econml 0.16.0 documentationeconml.azurewebsites.net
• Troubles with the Bias-Variance tradeoffejenner.com
• med.stanford.edu/content/dam/sm/dbds/documents/biostats-workshop/paper-1-.pdfmed.stanford.edu
• Machine Learning, Kolmogorov Complexity, and Squishy Bunniestheorangeduck.com
• [1707.01415] Machine Learning, Deepest Learning: Statistical Data Assimilation Problemsarxiv.org
• Hierarchical Modelingbetanalpha.github.io
• A Comprehensive Mechanistic Interpretability Explainer & Glossary — Neel Nandaneelnanda.io
• Overfitting in Machine Learning: What It Is and How to Prevent Itelitedatascience.com