Formation of Representations in Neural Networks

This is experimental HTML to improve accessibility. We invite you to report rendering errors. Use Alt+Y to toggle on accessible reporting links and Alt+Shift+Y to toggle off. Learn more about this project and help improve conversions. HTML conversions sometimes display errors due to content that did not convert correctly from the source. This paper uses the following packages that are not yet supported by the HTML conversion tool. Feedback on these issues are not necessary; they are known and are being worked on. Authors: achieve the best HTML results from your LaTeX submissions by following these best practices. Understanding neural representations will help open the black box of neural networks and advance our scientific understanding of modern AI systems. However, how complex, structured, and transferable representations emerge in modern neural networks has remained a mystery. Building on previous results, we propose the Canonical Representation Hypothesis (CRH), which posits a set

Formation of Representations in Neural Networks Formation of Representations in Neural Networks Liu Ziyin 1,3 , Isaac Chuang 1 , Tomer Galanti 2 , Tomaso Poggio 1 1 Massachusetts Institute of Technology 2 Texas A&M University 3 NTT Research Abstract Understanding neural representations will help open the black box of neural networks and advance our scientific understanding of modern AI systems. However, how complex, structured, and transferable representations emerge in modern neural networks has remained a mystery. Building on previous results, we propose the Canonical Representation Hypothes

related reading

• Toy Models of Superpositiontransformer-circuits.pub
• Zoom In: An Introduction to Circuitsdistill.pub
• The Platonic Representation Hypothesisphillipi.github.io
• The World Inside Neural Networksgoodfire.ai
• Neural Networks, Types, and Functional Programming -- colah's blogcolah.github.io
• Neural networks and deep learningneuralnetworksanddeeplearning.com
• [2604.21691] There Will Be a Scientific Theory of Deep Learningarxiv.org
• Understanding the Neural Tangent Kernel – EigenTaleseigentales.com
• The Platonic Representation Hypothesisarxiv.org
• Some Math behind Neural Tangent Kernel | Lil'Loglilianweng.github.io
• What Would Non-Linear Features Actually Look Like? — Liv Gortonlivgorton.com
• Statistical Mechanics of Deep Learningganguli-gang.stanford.edu