Model Tampering Attacks Enable More Rigorous Evaluations of LLM Capabilities

This is experimental HTML to improve accessibility. We invite you to report rendering errors. Use Alt+Y to toggle on accessible reporting links and Alt+Shift+Y to toggle off. Learn more about this project and help improve conversions. Evaluations of large language model (LLM) risks and capabilities are increasingly being incorporated into AI risk management and governance frameworks. Currently, most risk evaluations are conducted by designing inputs that elicit harmful behaviors from the system. However, this approach suffers from two limitations. First, input-output evaluations cannot fully evaluate realistic risks from open-weight models. Second, the behaviors identified during any particular input-output evaluation can only lower-bound the model’s worst-possible-case input-output behavior. As a complementary method for eliciting harmful behaviors, we propose evaluating LLMs with model tampering attacks which allow for modifications to latent activations or weights. We pit state-of-

Model Tampering Attacks Enable More Rigorous Evaluations of LLM Capabilities ∗ Zora Che, University of Maryland, ML Alignment & Theory Scholars zche@umd.edu ∗ Stephen Casper, MIT CSAIL, ML Alignment & Theory Scholars scasper@mit.edu Robert Kirk, UK AI Security Institute robert.kirk@dsit.gov.uk Anirudh Satheesh, University of Maryland anirudhs@terpmail.umd.edu Stewart Slocum, MIT slocumstewy@gmail.com Lev McKinney, University of Toronto levmckinney@cs.toronto.edu Rohit Gandikota, Northeastern University gandikota.ro@northeastern.edu Aidan Ewart, Haize Labs aidanprattewart@gmail.com Domenic Rosa

related reading

• Modifying LLM Beliefs with Synthetic Document Finetuningalignment.anthropic.com
• Your Evals Will Break and You Won't See It Coming - Lun Wangwanglun1996.github.io
• A small number of samples can poison LLMs of any size \ Anthropicanthropic.com
• Nicholas Carlininicholas.carlini.com
• Verifying your browser | OpenReviewopenreview.net
• [2312.06942] AI Control: Improving Safety Despite Intentional Subversionarxiv.org
• [2506.17209] Fine-Tuning Lowers Safety and Disrupts Evaluation Consistencyarxiv.org
• [2604.16812] Introspection Adapters: Training LLMs to Report Their Learned Behaviorsarxiv.org
• Adversarial Attacks on LLMs | Lil'Loglilianweng.github.io
• Introspection Adapters: Training LLMs to Report Their Learned Behaviorsarxiv.org
• Universal and Transferable Attacks on Aligned Language Modelsllm-attacks.org
• Deep Forgetting & Unlearning for Safely-Scoped LLMs — AI Alignment Forumalignmentforum.org