The Singular Value Decompositions of Transformer Weight Matrices are Highly Interpretable — LessWrong

Please go to the colab for interactive viewing and playing with the phenomena. For space reasons, not all results included in the colab are included…

x The Singular Value Decompositions of Transformer Weight Matrices are Highly Interpretable — LessWrong Interpretability (ML & AI) Conjecture (org) AI Frontpage 200 The Singular Value Decompositions of Transformer Weight Matrices are Highly Interpretable by beren , Sid Black 28th Nov 2022 AI Alignment Forum 37 min read 34 200 Ω 69 Please go to the colab for interactive viewing and playing with the phenomena. For space reasons, not all results included in the colab are included here so please visit the colab for the full story. A GitHub repository with the colab notebook and accompanying data c

related reading

• A Mathematical Framework for Transformer Circuitstransformer-circuits.pub
• Towards Monosemanticity: Decomposing Language Models With Dictionary Learningtransformer-circuits.pub
• interpreting GPT: the logit lens — LessWronglesswrong.com
• The Illustrated Transformer – Jay Alammar – Visualizing machine learning one concept at a time.jalammar.github.io
• Circuit Tracing: Revealing Computational Graphs in Language Modelstransformer-circuits.pub
• Verbalizable Representations Form a Global Workspace in Language Modelstransformer-circuits.pub
• Scaling Monosemanticity: Extracting Interpretable Features from Claude 3 Sonnettransformer-circuits.pub
• Towards Monosemanticity: Decomposing Language Models With Dictionary Learningtransformer-circuits.pub
• Softmax Linear Unitstransformer-circuits.pub
• The Annotated Transformernlp.seas.harvard.edu
• In-context Learning and Induction Headstransformer-circuits.pub
• Tensor-Transformer Variants are Surprisingly Performant — LessWronglesswrong.com