Sparse Attention Post-Training for Mechanistic Interpretability

N/A

# link_21s8412nms.pdf ## Metadata - PDFFormatVersion=1.7 - IsLinearized=false - IsAcroFormPresent=false - IsXFAPresent=false - IsCollectionPresent=false - IsSignaturesPresent=false - Author=Florent Draye; Anson Lei; Hsiao-Ru Pan; Ingmar Posner; Bernhard Schölkopf - Creator=arXiv GenPDF (tex2pdf:a6404ea) - Custom.DOI=https://doi.org/10.48550/arXiv.2512.05865 - Custom.License=http://creativecommons.org/licenses/by/4.0/ - Custom.PTEX.Fullbanner=This is pdfTeX, Version 3.141592653-2.6-1.40.28 (TeX Live 2025) kpathsea version 6.4.1 - Custom.arXivID=https://arxiv.org/abs/2512.05865v5 - Producer=pike

saved by

• Ali Ahmad

related reading

• A Mathematical Framework for Transformer Circuitstransformer-circuits.pub
• Weight-Sparse Circuits May Be Interpretable Yet Unfaithful — LessWronglesswrong.com
• An Extremely Opinionated Annotated List of My Favourite Mechanistic Interpretability Papers v2 — AI Alignment Forumalignmentforum.org
• Transformer Circuits Threadtransformer-circuits.pub
• Circuit Tracing: Revealing Computational Graphs in Language Modelstransformer-circuits.pub
• An Extremely Opinionated Annotated List of My Favourite Mechanistic Interpretability Papers v2 — AI Alignment Forumalignmentforum.org
• Towards Monosemanticity: Decomposing Language Models With Dictionary Learningtransformer-circuits.pub
• An Intuitive Explanation of Sparse Autoencoders for LLM Interpretability | Adam Karvonenadamkarvonen.github.io
• 2502.11089arxiv.org
• Sparse Autoencoders Find Highly Interpretable Features in Language Modelsarxiv.org
• Circuits Updates - January 2024transformer-circuits.pub
• Language Model Circuits Are Sparse in the Neuron Basis | Transluce AItransluce.org