Uncovering Neural Geometry in Vision Models With Block-Sparse Featurizers

We introduce Block-Sparse Featurizers (BSF), a family of methods to decompose a model’s activations into multidimensional subspaces rather than single directions. Applied to vision models, we find that BSFs find interpretable, multidimensional features which offer a more parsimonious explanation of model internals; that those features enable fine-grained steering; and that most concepts in the models are multidimensional.

Uncovering Neural Geometry in Vision Models With Block-Sparse Featurizers Research ← The Neural Geometry Series Uncovering Neural Geometry in Vision Models With Block-Sparse Featurizers We introduce Block-Sparse Featurizers (BSF), a family of methods to decompose a model's activations into multidimensional subspaces rather than single directions. Applied to vision models, we find that BSFs find interpretable, multidimensional features which offer a more parsimonious explanation of model internals; that those features enable fine-grained steering; and that most concepts in the models are multid

saved by

• Yixiong Hao
• Lydia Nottingham
• Sudarsh K

related reading

• Toy Models of Superpositiontransformer-circuits.pub
• Towards Monosemanticity: Decomposing Language Models With Dictionary Learningtransformer-circuits.pub
• When Models Manipulate Manifolds: The Geometry of a Counting Tasktransformer-circuits.pub
• The World Inside Neural Networksgoodfire.ai
• Feature Visualizationdistill.pub
• Towards Monosemanticity: Decomposing Language Models With Dictionary Learningtransformer-circuits.pub
• Matryoshka Sparse Autoencoders — LessWronglesswrong.com
• Softmax Linear Unitstransformer-circuits.pub
• The Building Blocks of Interpretabilitydistill.pub
• Prism: mapping interpretable concepts and features in a latent space of language | thesephist.comthesephist.com
• Feature-wise transformationsdistill.pub
• The Geometry of Concepts: Sparse Autoencoder Feature Structurearxiv.org