ceselder/dreaming-vectors: gradient-descented steering vectors from activation oracles
Don't get locked out of your account. Download your recovery codes or add a passkey so you don't lose access when you get a new device. There was an error while loading. Please reload this page. There was an error while loading. Please reload this page. There was an error while loading. Please reload this page. gradient-descented steering vectors from activation oracles There was an error while loading. Please reload this page. My MATS application! Full writeup on LessWrong Activation oracles (iterating on LatentQA) are an interpretability technique, capable of generating natural language explanations about activations with surprising generality. How robust are these oracles? Can we find a vector that maximises confidence of the oracle (in token probability) that a concept is represented, and can we then use this vector to steer the model? Is it possible to find feature representations that convince the oracle a concept is being represented, when it really is random noise? Eff
Dreaming Vectors: Gradient-descented steering vectors from Activation Oracles and using them to Red-Team AOs My MATS application! UPDATE: I got accepted! Full writeup on LessWrong TLDR Activation oracles (iterating on LatentQA) are an interpretability technique, capable of generating natural language explanations about activations with surprising generality. How robust are these oracles? Can we find a vector that maximises confidence of the oracle (in token probability) that a concept is represented, and can we then use this vector to steer the model? Is it possible to find feature representat
related reading
- Dreaming Vectors: Gradient-descented steering vectors from Activation Oracles and using them to Red-Team AOs — LessWronglesswrong.com
- Natural Language Autoencoders Produce Unsupervised Explanations of LLM Activationstransformer-circuits.pub
- I found >800 orthogonal “write code” steering vectors | Jacob’s Blogjacobgw.com
- Mechanistically Eliciting Latent Behaviors in Language Models — AI Alignment Forumalignmentforum.org
- Implementing activation steering — LessWronglesswrong.com
- Can activation verbalizers surface an internal chain of thought? — LessWronglesswrong.com
- Activation Steering in 2026: A Practitioner's Field Guide | Subhadip Mitrasubhadipmitra.com
- Scaling Activation Oracles to Trillion-Parameter Modelstransluce.org
- Attribution Patching: Activation Patching At Industrial Scale - Neel Nandaneelnanda.io
- Prism: mapping interpretable concepts and features in a latent space of language | thesephist.comthesephist.com
- Steering GPT-2-XL by adding an activation vector — AI Alignment Forumalignmentforum.org
- [2606.02609] Building Better Activation Oraclesarxiv.org