Anthropic Dictionary Learning Public Comment

See an accompanying colab tutorial to load the trained autoencoders, and interpret what the features do. One of the things I find most interesting about this paper is the existence of non neuron basis aligned features. One of the big mysteries (by my lights) in mechanistic interpretability is what the non-linearities in MLP layers are actually doing, on an algorithmic level. I can reason about monosemantic GELU neurons fairly easily (like a French neuron) - essentially thinking of it as a soft ReLU, that collect pieces of evidence for the presence of a feature, and fire if they cross a certain threshold (given by the bias). This can maybe extend to thinking about a sparse linear combination of neurons (eg fewer than 10 constructively interfering to create a single feature). But I have no idea how to reason about things that are dense-ish in the neuron basis! As a first step towards exploring this, I looked into how dense vs sparse each non-ultra-low frequency feature was. Conceptually,

Anthropic Dictionary Learning Public Comment Neel Nanda is an external mechanistic interpretability researcher. See an accompanying colab tutorial to load the trained autoencoders, and interpret what the features do. TLDR The core results seem to replicate - I trained a sparse autoencoder on the MLP layer of an open source 1 layer GELU language model, and a significant fraction of the latent space features were interpretable I open source two trained autoencoders, here’s a tutorial for how to use them, and how to interpret a feature . A nd a (very!) rough training codebase I exhibit some case