3.1 Importance of Interpretability | Interpretable Machine Learning

Machine learning algorithms usually operate as black boxes and it is unclear how they derived a certain decision. This book is a guide for practitioners to make machine learning decisions interpretable.

This chapter introduces the concepts of interpretability. While it’s difficult to define interpretability mathematically, I like the definition by Biran and Cotton (2017), which was also used by Miller (2019): “Interpretability is the degree to which a human can understand the cause of a decision.” Another good one is by Kim, Khanna, and Koyejo (2016): “a method is interpretable if a user can correctly and efficiently predict the method’s results” The more interpretable a machine learning model, the easier it is for someone to understand why certain decisions or predictions were made. A…

related reading

• Against Interpretability: a Critical Examination of the Interpretability Problem in Machine Learninglink.springer.com
• The Hitchhiker's Guide to Actionable Interpretabilityactionable-interpretability-guide.github.io
• 6 – Interpretability – Machine Learning Blog | ML@CMU | Carnegie Mellon Universityblog.ml.cmu.edu
• The Building Blocks of Interpretabilitydistill.pub
• On Optimism for Interpretabilitygoodfire.ai
• Faithful, Interpretable Model Explanations via Causal Abstraction | SAIL Blogai.stanford.edu
• In (highly contingent!) defense of interpretability-in-the-loop ML training — AI Alignment Forumalignmentforum.org
• How To Become A Mechanistic Interpretability Researcher — AI Alignment Forumalignmentforum.org
• An Intuitive Explanation of Sparse Autoencoders for LLM Interpretability | Adam Karvonenadamkarvonen.github.io
• What is the purpose of interpretability?ericjmichaud.com
• Principles and Practice of Explainable Machine Learningfrontiersin.org
• [1811.10154] Stop Explaining Black Box Machine Learning Models for High Stakes Decisions and Use Interpretable Models Insteadarxiv.org