Understanding the Parameter Decomposition papers
Before reading this post, I recommend reading Lee Sharkey's post "Mech Interp is not Pre-paradigmatic." I think the post is worthwhile in itself as a survey of mech interp methods and history; it also explains Parameter Decomposition's basic ideas. Ideally you will have familiarity with those before reading this post, because I don't spend a ton of time outlining them. This is mostly an explanation of the technical details behind what Goodfire and Sharkey are shooting for with Parameter Decomposition. This post closely follows the content of the APD paper Interpretability in Parameter Space: Minimizing Mechanistic Description Length with Attribution-based Parameter Decomposition and the SPD paper Stochastic Parameter Decomposition. I follow the notation there, but I sometimes use my own structure. To recap, PD takes all the weight matrices of the model and flattens them into a vector in 'parameter-space'. Then it aims to turn this vector into the sum of a bunch of other, simpler weight
404 Error 404 Not Found There's nothing at this URL :( (try searching?)
Explore this link on the map →related reading
- Attribution-based parameter decomposition — LessWronglesswrong.com
- Interpreting Language Model Parametersgoodfire.ai
- Stochastic Parameter Decompositionarxiv.org
- Towards Scalable Parameter Decompositiongoodfire.ai
- [2501.14926] Interpretability in Parameter Space: Minimizing Mechanistic Description Length with Attribution-based Parameter Decompositionarxiv.org
- [2506.20790] Stochastic Parameter Decompositionarxiv.org
- An Extremely Opinionated Annotated List of My Favourite Mechanistic Interpretability Papers v2 — AI Alignment Forumalignmentforum.org
- Towards Scalable Parameter Decompositiongoodfire.ai
- An Extremely Opinionated Annotated List of My Favourite Mechanistic Interpretability Papers v2 — AI Alignment Forumalignmentforum.org
- How To Become A Mechanistic Interpretability Researcher — AI Alignment Forumalignmentforum.org
- Mech interp is not pre-paradigmatic — LessWronglesswrong.com
- Transformer Circuits Threadtransformer-circuits.pub