What Would Non-Linear Features Actually Look Like? | Liv Gorton
“Non-linear representations” have become a catch-all objection to mechanistic interpretability work. The concern is worth taking seriously, but as typically stated, it collapses together cases with completely different implications and likelihoods.
What Would Non-Linear Features Actually Look Like? January 27, 2026 / 15 min read “Non-linear representations” have become a catch-all objection to mechanistic interpretability work. The concern is worth taking seriously, but as typically stated, it collapses together cases with completely different implications and likelihoods. Debates about non-linear features have often been unconstructive for several reasons. One common issue is that people have used the word “linear feature” to mean multiple different things, leading to miscommunication. Given this, the first contribution of this post is
saved by
related reading
- Toy Models of Superpositiontransformer-circuits.pub
- What if not Circuits? — LessWronglesswrong.com
- Feature Visualizationdistill.pub
- Neural networks are not already cleanlesswrong.com
- Softmax Linear Unitstransformer-circuits.pub
- [2605.01192] Linear-Readout Floors and Threshold Recovery in Computation in Superpositionarxiv.org
- An Extremely Opinionated Annotated List of My Favourite Mechanistic Interpretability Papers v2 — AI Alignment Forumalignmentforum.org
- An Extremely Opinionated Annotated List of My Favourite Mechanistic Interpretability Papers v2 — AI Alignment Forumalignmentforum.org
- A short note on interpretability and mindsericjmichaud.com
- Interpreting Language Model Parametersgoodfire.ai
- The Building Blocks of Interpretabilitydistill.pub
- A Comprehensive Mechanistic Interpretability Explainer & Glossary — Neel Nandaneelnanda.io