Sieve: SAEs Beat Baselines on a Real-World Task (A Code Generation Case Study) | Tilde

Our methods achieve Pareto dominance on the axis of task success rate vs task constraint satisfaction vs general model performance. Mechanistic interpretability aims to explain the internal mechanisms of models from the weights of the model alone. However, this goal is complicated by behavior like superposition[1]. In superposition, rather than each neuron representing one set of semantic features, dissimilar features emerge from combinations of neurons firing together. One promising way of overcoming this is via sparse autoencoders (SAEs)[2], a fully unsupervised approach whereby one learns a large linear “dictionary” of monosemantic, interpretable features that exist in the model. The key idea is that by encouraging sparsity in the right way, we can obtain more interpretable and meaningful features. Concretely, SAEs learn a dictionary of encoder and decoder vectors that first map representations in the model’s activation space (its “information highway”) to a sparse latent space of f

Our methods achieve Pareto dominance on the axis of task success rate vs task constraint satisfaction vs general model performance. Mechanistic interpretability aims to explain the internal mechanisms of models from the weights of the model alone. However, this goal is complicated by behavior like superposition[1]. In superposition, rather than each neuron representing one set of semantic features, dissimilar features emerge from combinations of neurons firing together. One promising way of overcoming this is via sparse autoencoders (SAEs)[2], a fully unsupervised approach whereby one learns a