I found >800 orthogonal “write code” steering vectors | Jacob’s Blog
A few weeks ago, I stumbled across a very weird fact: it is possible to find multiple steering vectors in a language model that activate very similar behaviors while all being orthogonal. This was pretty surprising to me and to some people that I talked to, so I decided to write a post about it. I don’t currently have the bandwidth to investigate this much more, so I’m just putting this post and the code up. I’ll first discuss how I found these orthogonal steering vectors, then share some results. Finally, I’ll discuss some possible explanations for what is happening. My work here builds upon Mechanistically Eliciting Latent Behaviors in Language Models (MELBO). I use MELBO to find steering vectors. Once I have a MELBO vector, I then use my algorithm to generate vectors orthogonal to it that do similar things. Define f ( x ) 𝑓 ( 𝑥 ) as the activation-activation map that takes as input layer 8 activations of the language model and returns layer 16 activations after being passed thro
Produced as part of the MATS Summer 2024 program, under the mentorship of Alex Turner. A few weeks ago, I stumbled across a very weird fact: it is possible to find multiple steering vectors in a language model that activate very similar behaviors while all being orthogonal . This was pretty surprising to me and to some people that I talked to, so I decided to write a post about it. I don’t currently have the bandwidth to investigate this much more, so I’m just putting this post and the code up. I’ll first discuss how I found these orthogonal steering vectors, then share some results. Finally,
saved by
related reading
- I found >800 orthogonal "write code" steering vectors — LessWronglesswrong.com
- Dreaming Vectors: Gradient-descented steering vectors from Activation Oracles and using them to Red-Team AOs — LessWronglesswrong.com
- Mechanistically Eliciting Latent Behaviors in Language Models — AI Alignment Forumalignmentforum.org
- Tracing the thoughts of a large language model \ Anthropicanthropic.com
- Activation Steering in 2026: A Practitioner's Field Guide | Subhadip Mitrasubhadipmitra.com
- Verbalizable Representations Form a Global Workspace in Language Modelstransformer-circuits.pub
- Neuronpedianeuronpedia.org
- Refusal in LLMs is mediated by a single direction — LessWronglesswrong.com
- Can activation verbalizers surface an internal chain of thought? — LessWronglesswrong.com
- Scaling Monosemanticity: Extracting Interpretable Features from Claude 3 Sonnettransformer-circuits.pub
- Natural Language Autoencoders Produce Unsupervised Explanations of LLM Activationstransformer-circuits.pub
- Transformer Circuits Threadtransformer-circuits.pub