✳flâneur — a map of the web's best reading

Sparse Crosscoders for Cross-Layer Features and Model Diffing

transformer-circuits.pub · 20,040 words · saved by 1 readers

This note introduces sparse crosscoders, a variant of sparse autoencoders (e.g. ) or transcoders for understanding models in superposition . Where autoencoders encode and predict activations at a single layer, and transcoders use activations from one layer to predict the next, a crosscoder reads and writes to multiple layers. Crosscoders produce shared features across layers and even models. They have several applications: This note will cover some theoretical examples motivating crosscoders, and then present preliminary experiments applying them to cross-layer superposition and model diffing. We also briefly discuss the theory of how crosscoders might simplify circuit analysis, but leave results on this for a future update. According to the superposition hypothesis, neural networks represent more features than they have neurons by allowing features to be non-orthogonal . One consequence of this is that most features are represented by linear combinations of multiple neurons: At first

Sparse Crosscoders for Cross-Layer Features and Model Diffing Transformer Circuits Thread Sparse Crosscoders for Cross-Layer Features and Model Diffing Sparse Crosscoders for Cross-Layer Features and Model Diffing Authors Jack Lindsey * , Adly Templeton * , Jonathan Marcus * , Thomas Conerly * , Joshua Batson, Christopher Olah Affiliations Anthropic Published Oct 25, 2024 * Core Contributor Research Update: This preliminary note is a research update, similar to our monthly updates (albeit larger). We'd ask you to treat it in the spirit of results being presented at a lab meeting or internal se

Explore this link on the map →

Sparse Crosscoders for Cross-Layer Features and Model Diffing

related reading