What Is The True Name of Modularity? — LessWrong
TL;DR Modularity seems like an important feature of neural networks, but there is currently no canonical way of properly defining or measuring it whi…
x What Is The True Name of Modularity? — LessWrong Selection Theorems: Modularity Modularity Causality Information theory World Modeling AI Frontpage 41 What Is The True Name of Modularity? by CallumMcDougall , Lucius Bushnaq , Avery 1st Jul 2022 AI Alignment Forum 14 min read 10 41 Ω 16 TL;DR Modularity seems like an important feature of neural networks, but there is currently no canonical way of properly defining or measuring it which is properly theoretically motivated and doesn’t break down in some cases - in other words, we haven’t yet found a True Name for it. Most modularity measures us
related reading
- Zoom In: An Introduction to Circuitsdistill.pub
- Visual Information Theory -- colah's blogcolah.github.io
- On neural scaling and the quanta hypothesisericjmichaud.com
- The Building Blocks of Interpretabilitydistill.pub
- Investigating the learning coefficient of modular addition: hackathon project — LessWronglesswrong.com
- Modularity of Mindplato.stanford.edu
- Evolution of Modularity — LessWronglesswrong.com
- [2410.21265] Modular Duality in Deep Learningarxiv.org
- Interpreting Language Model Parametersgoodfire.ai
- Multimodal Neurons in Artificial Neural Networksdistill.pub
- Fact Finding: Attempting to Reverse-Engineer Factual Recall on the Neuron Level (Post 1) — AI Alignment Forumalignmentforum.org
- [2103.03386] Clusterability in Neural Networksarxiv.org