flâneur

Towards Scalable Parameter Decomposition

goodfire.ai · 1,832 words · saved by 1 readers

In 2013, researchers discovered that word2vec embeddings encoded analogies in their parameters: the now-famous result that vector arithmetic like king - man + woman = queen worked directly in embedding space. A decade later, we're still better at interpreting the information that flows through neural networks than understanding the machinery itself. The mechanistic interpretability community has made real progress decomposing neural networks into understandable parts—we can identify representations, trace circuits, even edit some specific behaviors. But the most successful methods so far, like SAEs, have focused largely on the activations that flow through a model, rather than directly inspecting the weights that transform and guide the inputs into those flows. It's a bit like trying to understand a program by only looking at its runtime variables, but never its source code. This matters because we're ultimately after mechanistic stories—causal explanations of how models implement thei

Towards Scalable Parameter Decomposition Research Towards Scalable Parameter Decomposition Authors Lucius Bushnaq *† Dan Braun *† Lee Sharkey ‡ * Co-first Author † Goodfire – Work primarily carried out while at Apollo Research ‡ Goodfire Correspondence to lucius@goodfire.ai Blog post by Michael Byun Published June 27, 2025 Full Paper Read on arXiv → Like a game of Jenga, SPD finds parameter components that can be removed while keeping the model's behavior stable. "Jenga" by Sheila Sund, licensed under CC BY 2.0 . In 2013, researchers discovered that word2vec embeddings encoded analogies in the

saved by

related reading