[2506.10805] Detecting High-Stakes Interactions with Activation Probes

Abstract:Monitoring is an important aspect of safely deploying Large Language Models (LLMs). This paper examines activation probes for detecting ``high-stakes'' interactions -- where the text indicates that the interaction might lead to significant harm -- as a critical, yet underexplored, target for such monitoring. We evaluate several probe architectures trained on synthetic data, and find them to exhibit robust generalization to diverse, out-of-distribution, real-world data. Probes' performance is comparable to that of prompted or finetuned medium-sized LLM monitors, while offering computational savings of six orders-of-magnitude. These savings are enabled by reusing activations of the model that is being monitored. Our experiments also highlight the potential of building resource-aware hierarchical monitoring systems, where probes serve as an efficient initial filter and flag cases for more expensive downstream analysis. We release our novel synthetic dataset and the codebase at this https URL.

Detecting High-Stakes Interactions with Activation Probes Alex McKenzie1∗ Urja Pawar1∗ Phil Blandfort1∗ William Bankes1,2∗ David Krueger3 Ekdeep Singh Lubana4,5,6 Dmitrii Krasheninnikov7 1 LASR Labs 2 University College London 3 MILA…

related reading

• [2506.10805] Detecting High-Stakes Interactions with Activation Probesarxiv.org
• [2604.16812] Introspection Adapters: Training LLMs to Report Their Learned Behaviorsarxiv.org
• [2512.11949] Neural Chameleons: Language Models Can Learn to Hide Their Thoughts from Unseen Activation Monitorsarxiv.org
• Surfacing Pathological Behaviors in Language Models | Transluce AItransluce.org
• [2507.11473] Chain of Thought Monitorability: A New and Fragile Opportunity for AI Safetyarxiv.org
• Probe-Based Data Attribution: Surfacing and Mitigating Undesirable Behaviors in LLM Post-Traininggoodfire.ai
• [2608.21664] Measuring Activation Control in Large Language Modelsarxiv.org
• [2502.05209] Model Tampering Attacks Enable More Rigorous Evaluations of LLM Capabilitiesarxiv.org
• 2308.03958arxiv.org
• [2503.10965] Auditing language models for hidden objectivesarxiv.org
• [2607.14111] Introspection Fine-Tuning (IFT): Training Small LLMs to Introspectarxiv.org
• Probe-Based Data Attribution: Surfacing and Mitigating Undesirable Behaviors in LLM Post-Training - Goodfiregoodfire.com