Against Interpretability: a Critical Examination of the Interpretability Problem in Machine Learning | Philosophy & Technology | Springer Nature Link
The usefulness of machine learning algorithms has led to their widespread adoption prior to the development of a conceptual framework for making sense of them. One common response to this situation is to say that machine learning suffers from a “black box problem.” That is, machine learning algorithms are “opaque” to human users, failing to be “interpretable” or “explicable” in terms that would render categorization procedures “understandable.” The purpose of this paper is to challenge the widespread agreement about the existence and importance of a black box problem. The first section argues that “interpretability” and cognates lack precise meanings when applied to algorithms. This makes the concepts difficult to use when trying to solve the problems that have motivated the call for interpretability (etc.). Furthermore, since there is no adequate account of the concepts themselves, it is not possible to assess whether particular technical features supply formal definitions of those concepts. The second section argues that there are ways of being a responsible user of these algorithms that do not require interpretability (etc.). In many cases in which a black box problem is cited, interpretability is a means to a further end such as justification or non-discrimination. Since addressing these problems need not involve something that looks like an “interpretation” (etc.) of an algorithm, the focus on interpretability artificially constrains the solution space by characterizing one possible solution as the problem itself. Where possible, discussion should be reformulated in terms of the ends of interpretability.
1 Introduction The usefulness of machine learning (ML) algorithms has led to their adoption prior to the development of a conceptual framework for making sense of them. When people and organizations use tools that they do not understand, they risk failures that jeopardize lives, as well as more subtle and invidious failures such as promoting discriminatory outcomes. Growing awareness of such risks has made it commonplace to say that machine learning suffers from a “black box problem.” The basic statement of the problem is this: Although classifiers built by new machine learning techniques…
saved by
related reading
- What is the purpose of interpretability?ericjmichaud.com
- 2 Interpretability – Interpretable Machine Learningchristophm.github.io
- Dario Amodei — The Urgency of Interpretabilitydarioamodei.com
- [1811.10154] Stop Explaining Black Box Machine Learning Models for High Stakes Decisions and Use Interpretable Models Insteadarxiv.org
- 6 – Interpretability – Machine Learning Blog | ML@CMU | Carnegie Mellon Universityblog.ml.cmu.edu
- The Hitchhiker's Guide to Actionable Interpretabilityactionable-interpretability-guide.github.io
- Martian Interpretability Challenge: The Core Problems In Interpretability — LessWronglesswrong.com
- Neel Nanda on Mechanistic Interpretability: Progress, Limits, and Paths to Safer AI — EA Forumforum.effectivealtruism.org
- A Pragmatic Vision for Interpretability — AI Alignment Forumalignmentforum.org
- An Ambitious Vision for Interpretability — AI Alignment Forumalignmentforum.org
- In (highly contingent!) defense of interpretability-in-the-loop ML training — AI Alignment Forumalignmentforum.org
- How To Become A Mechanistic Interpretability Researcher — AI Alignment Forumalignmentforum.org