Branch Specialization
distill.pub · 3,390 words · saved by 1 readers
When a neural network layer is divided into multiple branches, neurons self-organize into coherent groupings.
Branch Specialization Distill Branch Specialization Authors Affiliations Chelsea Voss OpenAI Gabriel Goh OpenAI Nick Cammarata OpenAI Michael Petrov OpenAI Ludwig Schubert Chris Olah Published April 5, 2021 DOI 10.23915/distill.00024.008 This article is part of the Circuits thread , an experimental format collecting invited short articles and critical commentary delving into the inner workings of neural networks. Visualizing Weights Weight Banding Introduction If we think of interpretability as a kind of “anatomy of neural networks,” most of the circuits thread has involved studying tiny littl
related reading
- Toy Models of Superpositiontransformer-circuits.pub
- Naturally Occurring Equivariance in Neural Networksdistill.pub
- Visualizing Weightsdistill.pub
- Zoom In: An Introduction to Circuitsdistill.pub
- Feature Visualizationdistill.pub
- Neural Networks, Manifolds, and Topology -- colah's blogcolah.github.io
- Neural networks and deep learningneuralnetworksanddeeplearning.com
- The Building Blocks of Interpretabilitydistill.pub
- Multimodal Neurons in Artificial Neural Networksdistill.pub
- Feature Visualizationdistill.pub
- [2103.03386] Clusterability in Neural Networksarxiv.org
- What if not Circuits? — LessWronglesswrong.com