[2312.06681] Steering Llama 2 via Contrastive Activation Addition

Abstract:We introduce Contrastive Activation Addition (CAA), an innovative method for steering language models by modifying their activations during forward passes. CAA computes "steering vectors" by averaging the difference in residual stream activations between pairs of positive and negative examples of a particular behavior, such as factual versus hallucinatory responses. During inference, these steering vectors are added at all token positions after the user's prompt with either a positive or negative coefficient, allowing precise control over the degree of the targeted behavior. We evaluate CAA's effectiveness on Llama 2 Chat using multiple-choice behavioral question datasets and open-ended generation tasks. We demonstrate that CAA significantly alters model behavior, is effective over and on top of traditional methods like finetuning and system prompt design, and minimally reduces capabilities. Moreover, we gain deeper insights into CAA's mechanisms by employing various activation space interpretation methods. CAA accurately steers model outputs and sheds light on how high-level concepts are represented in Large Language Models (LLMs).

Abstract:We introduce Contrastive Activation Addition (CAA), an innovative method for steering language models by modifying their activations during forward passes. CAA computes "steering vectors" by averaging the difference in residual stream activations between pairs of positive and negative examples of a particular behavior, such as factual versus hallucinatory responses. During inference, these steering vectors are added at all token positions after the user's prompt with either a positive or negative coefficient, allowing precise control over the degree of the targeted behavior. We evalua

related reading

• Steering Llama-2 with contrastive activation additions — LessWronglesswrong.com
• On the Biology of a Large Language Modeltransformer-circuits.pub
• Natural Language Autoencoders Produce Unsupervised Explanations of LLM Activationstransformer-circuits.pub
• Steering GPT-2-XL by adding an activation vector — AI Alignment Forumalignmentforum.org
• Activation Steering in 2026: A Practitioner's Field Guide | Subhadip Mitrasubhadipmitra.com
• Steering Might Stop Working Soon — LessWronglesswrong.com
• Llama 2 · Hugging Facehuggingface.co
• Chapter 4: Alignment Science - ARENAlearn.arena.education
• GitHub - PaulPauls/llama3_interpretability_sae: A complete end-to-end pipeline for LLM interpretability with sparse autoencoders (SAEs) using Llama 3.2, written in pure PyTorch and fully reproducible. · GitHubgithub.com
• Emotion Concepts and their Function in a Large Language Modeltransformer-circuits.pub
• Steering Black-Box LLMs with Advisor Modelsarxiv.org
• Neuronpedianeuronpedia.org