Do Not Rely on the Linear Representation Hypothesis
We learn a single steering vector that steers for 4 different tasks when applied at 4 different coefficients. The core result of this note is that you can learn a single steering vector that steers for 4 four different tasks when applied at four different norms. You can also learn two steering vectors that steer well for a task each, but when summed, steer for a completely unrelated task. Based on these experiments, I encourage interp and safety practitioners not to rely on the linear representation hypothesis: that directions in activation space correspond to high-level features (or, concepts) used by a network. I write about this because linearity-inspired hypotheses are common in interp thought, not just as falsifiable hypotheses, but as core aesthetic and intuition-building ideas. There's evidence: additive steering works well! Linear probes work well. Counterexamples may seem like either richer but linearly decodable structure (like rings for days of the week; Engels et al., 2025)