[2406.11717] Refusal in Language Models Is Mediated by a Single Direction

arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website. Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them. Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs. arXiv Operational Status Get status notifications via email or slack

[2406.11717] Refusal in Language Models Is Mediated by a Single Direction Skip to main content arXiv is now an independent nonprofit! Learn more × Search arXiv Press Enter to search · Advanced search --> Computer Science > Machine Learning arXiv:2406.11717 (cs) [Submitted on 17 Jun 2024 ( v1 ), last revised 30 Oct 2024 (this version, v3)] Title: Refusal in Language Models Is Mediated by a Single Direction Authors: Andy Arditi , Oscar Obeso , Aaquib Syed , Daniel Paleka , Nina Panickssery , Wes Gurnee , Neel Nanda View a PDF of the paper titled Refusal in Language Models Is Mediate

related reading

• Refusal in LLMs is mediated by a single direction — LessWronglesswrong.com
• Tracing the thoughts of a large language model \ Anthropicanthropic.com
• How confessions can keep language models honest | OpenAIopenai.com
• Discovering Language Model Behaviors with Model-Written Evaluations — LessWronglesswrong.com
• [2203.02155] Training language models to follow instructions with human feedbackarxiv.org
• Externalized reasoning oversight: a research direction for language model alignment — AI Alignment Forumalignmentforum.org
• Surfacing Pathological Behaviors in Language Models | Transluce AItransluce.org
• Thought Branches: Interpreting LLM Reasoning Requires Resamplingarxiv.org
• Red Teaming Language Models with Language Models — Google DeepMinddeepmind.com
• Mechanistically Eliciting Latent Behaviors in Language Models — AI Alignment Forumalignmentforum.org
• Mechanistically Eliciting Latent Behaviors in Language Models — LessWronglesswrong.com
• [2601.10387] The Assistant Axis: Situating and Stabilizing the Default Persona of Language Modelsarxiv.org