Why advanced machine learning methods badly overfit niche models – is this statistical machismo? | Dynamic Ecology

Statistical machismo, as I’ve been defining it in this blog (here and here) is when there is a push to ever newer, more complex methods, often in pursuit of a one-dimensional form of improvem…

Statistical machismo, as I’ve been defining it in this blog ( here and here ) is when there is a push to ever newer, more complex methods, often in pursuit of a one-dimensional form of improvement while ignoring the fact that choosing the proper statistic involves complex trade-offs. For example, hyperfocusing on one source of error while ignoring (or even worsening) other sources of error. Here I want to talk about how the new and the complex sides of statistical machismo interact badly in the most common application of machine learning to ecology to date: niche models (aka species dist

related reading

• Too much efficiency makes everything worse: overfitting and the strong version of Goodhart’s law | Jascha’s blogsohl-dickstein.github.io
• A statistical approach to model evaluations \ Anthropicanthropic.com
• Overfitting in Machine Learning: What It Is and How to Prevent Itelitedatascience.com
• Preface - The Emerging Science of Machine Learning Benchmarksmlbenchmarks.org
• 3.6. Generalization — Dive into Deep Learning 1.0.3 documentationd2l.ai
• deeplearningbook.org/contents/ml.htmldeeplearningbook.org
• Researchbactra.org
• Troubles with the Bias-Variance tradeoffejenner.com
• Hierarchical Modelingbetanalpha.github.io
• Rediscovery of Interiority in MLdanburfoot.net
• 4 – The Overfitting Iceberg – Machine Learning Blog | ML@CMU | Carnegie Mellon Universityblog.ml.cmu.edu
• [2108.02497] How to avoid machine learning pitfalls: a guide for academic researchersarxiv.org