I found >800 orthogonal “write code” steering vectors | Jacob’s Blog

A few weeks ago, I stumbled across a very weird fact: it is possible to find multiple steering vectors in a language model that activate very similar behaviors while all being orthogonal. This was pretty surprising to me and to some people that I talked to, so I decided to write a post about it. I don’t currently have the bandwidth to investigate this much more, so I’m just putting this post and the code up. I’ll first discuss how I found these orthogonal steering vectors, then share some results. Finally, I’ll discuss some possible explanations for what is happening. My work here builds upon Mechanistically Eliciting Latent Behaviors in Language Models (MELBO). I use MELBO to find steering vectors. Once I have a MELBO vector, I then use my algorithm to generate vectors orthogonal to it that do similar things. Define f ( x ) 𝑓 ( 𝑥 ) as the activation-activation map that takes as input layer 8 activations of the language model and returns layer 16 activations after being passed thro

Produced as part of the MATS Summer 2024 program, under the mentorship of Alex Turner. A few weeks ago, I stumbled across a very weird fact: it is possible to find multiple steering vectors in a language model that activate very similar behaviors while all being orthogonal . This was pretty surprising to me and to some people that I talked to, so I decided to write a post about it. I don’t currently have the bandwidth to investigate this much more, so I’m just putting this post and the code up. I’ll first discuss how I found these orthogonal steering vectors, then share some results. Finally,

saved by

• Lydia Nottingham
• Miles Kodama
• Malaika Aiyar

related reading

• I found >800 orthogonal "write code" steering vectors — LessWronglesswrong.com
• Dreaming Vectors: Gradient-descented steering vectors from Activation Oracles and using them to Red-Team AOs — LessWronglesswrong.com
• Tracing the thoughts of a large language model \ Anthropicanthropic.com
• Activation Steering in 2026: A Practitioner's Field Guide | Subhadip Mitrasubhadipmitra.com
• Verbalizable Representations Form a Global Workspace in Language Modelstransformer-circuits.pub
• Neuronpedianeuronpedia.org
• Scaling Monosemanticity: Extracting Interpretable Features from Claude 3 Sonnettransformer-circuits.pub
• Can activation verbalizers surface an internal chain of thought? — LessWronglesswrong.com
• Transformer Circuits Threadtransformer-circuits.pub
• Natural Language Autoencoders Produce Unsupervised Explanations of LLM Activationstransformer-circuits.pub
• Steering GPT-2-XL by adding an activation vector — AI Alignment Forumalignmentforum.org
• Discovering Language Model Behaviors with Model-Written Evaluations — LessWronglesswrong.com