[2505.14352] Towards eliciting latent knowledge from LLMs with mechanistic interpretability

Abstract:As language models become more powerful and sophisticated, it is crucial that they remain trustworthy and reliable. There is concerning preliminary evidence that models may attempt to deceive or keep secrets from their operators. To explore the ability of current techniques to elicit such hidden knowledge, we train a Taboo model: a language model that describes a specific secret word without explicitly stating it. Importantly, the secret word is not presented to the model in its training data or prompt. We then investigate methods to uncover this secret. First, we evaluate non-interpretability (black-box) approaches. Subsequently, we develop largely automated strategies based on mechanistic interpretability techniques, including logit lens and sparse autoencoders. Evaluation shows that both approaches are effective in eliciting the secret word in our proof-of-concept setting. Our findings highlight the promise of these approaches for eliciting hidden knowledge and suggest several promising avenues for future work, including testing and refining these methods on more complex model organisms. This work aims to be a step towards addressing the crucial problem of eliciting secret knowledge from language models, thereby contributing to their safe and reliable deployment.

Abstract:As language models become more powerful and sophisticated, it is crucial that they remain trustworthy and reliable. There is concerning preliminary evidence that models may attempt to deceive or keep secrets from their operators. To explore the ability of current techniques to elicit such hidden knowledge, we train a Taboo model: a language model that describes a specific secret word without explicitly stating it. Importantly, the secret word is not presented to the model in its training data or prompt. We then investigate methods to uncover this secret. First, we evaluate non-interpr

related reading

• Tracing the thoughts of a large language model \ Anthropicanthropic.com
• Transformer Circuits Threadtransformer-circuits.pub
• Mapping the Mind of a Large Language Model \ Anthropicanthropic.com
• An Extremely Opinionated Annotated List of My Favourite Mechanistic Interpretability Papers v2 — AI Alignment Forumalignmentforum.org
• [2401.05566] Sleeper Agents: Training Deceptive LLMs that Persist Through Safety Trainingarxiv.org
• Neel Nanda on Mechanistic Interpretability: Progress, Limits, and Paths to Safer AI — EA Forumforum.effectivealtruism.org
• Mechanistic interpretability: 10 Breakthrough Technologies 2026 | MIT Technology Reviewtechnologyreview.com
• Prediction, Explanation, or Over-interpretation?elena-baixy.github.io
• Eliciting Language Model Behaviors with Investigator Agents | Transluce AItransluce.org
• Mechanistically Eliciting Latent Behaviors in Language Models — AI Alignment Forumalignmentforum.org
• Neural Chameleons: LLMs Can Learn to Evade Activation Monitorsneuralchameleons.com
• Chapter 1: Transformer Interpretability - ARENAlearn.arena.education