[2504.13151] MIB: A Mechanistic Interpretability Benchmark

arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website. Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them. Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs. arXiv Operational Status Get status notifications via email or slack

[2504.13151] MIB: A Mechanistic Interpretability Benchmark Skip to main content arXiv is now an independent nonprofit! Learn more × Search arXiv Press Enter to search · Advanced search --> Computer Science > Machine Learning arXiv:2504.13151 (cs) [Submitted on 17 Apr 2025 ( v1 ), last revised 9 Jun 2025 (this version, v2)] Title: MIB: A Mechanistic Interpretability Benchmark Authors: Aaron Mueller , Atticus Geiger , Sarah Wiegreffe , Dana Arad , Iván Arcuschin , Adam Belfki , Yik Siu Chan , Jaden Fiotto-Kaufman , Tal Haklay , Michael Hanna , Jing Huang , Rohan Gupta , Yaniv Nikank

related reading

• [2404.19756] KAN: Kolmogorov-Arnold Networksarxiv.org
• [2406.17583] Towards Compositional Interpretability for XAIarxiv.org
• [2406.11717] Refusal in Language Models Is Mediated by a Single Directionarxiv.org
• Mechanistic Interpretability of Binary and Ternary Transformersarxiv.org
• [2512.19428] Attention Is Not What You Needarxiv.org
• Towards Automated Circuit Discovery for Mechanistic Interpretabilityarxiv.org
• [2402.12875] Chain of Thought Empowers Transformers to Solve Inherently Serial Problemsarxiv.org
• Interactive Machine Learning: A State of the Art Reviewarxiv.org
• [2406.02061] Alice in Wonderland: Simple Tasks Showing Complete Reasoning Breakdown in State-Of-the-Art Large Language Modelsarxiv.org
• [2305.18654] Faith and Fate: Limits of Transformers on Compositionalityarxiv.org
• [2401.05654] Towards Conversational Diagnostic AIarxiv.org
• [2503.03750] The MASK Benchmark: Disentangling Honesty From Accuracy in AI Systemsarxiv.org