What Is The True Name of Modularity? — LessWrong
TL;DR Modularity seems like an important feature of neural networks, but there is currently no canonical way of properly defining or measuring it whi…
x What Is The True Name of Modularity? — LessWrong Selection Theorems: Modularity Modularity Causality Information theory World Modeling AI Frontpage 41 What Is The True Name of Modularity? by CallumMcDougall , Lucius Bushnaq , Avery 1st Jul 2022 AI Alignment Forum 14 min read 10 41 Ω 16 TL;DR Modularity seems like an important feature of neural networks, but there is currently no canonical way of properly defining or measuring it which is properly theoretically motivated and doesn’t break down in some cases - in other words, we haven’t yet found a True Name for it. Most modularity measures us
Explore this link on the map →related reading
- Zoom In: An Introduction to Circuitsdistill.pub
- Visual Information Theory -- colah's blogcolah.github.io
- On neural scaling and the quanta hypothesisericjmichaud.com
- The Building Blocks of Interpretabilitydistill.pub
- Investigating the learning coefficient of modular addition: hackathon project — LessWronglesswrong.com
- Evolution of Modularity — LessWronglesswrong.com
- [2410.21265] Modular Duality in Deep Learningarxiv.org
- Attribution Patching: Activation Patching At Industrial Scale - Neel Nandaneelnanda.io
- Interpreting Language Model Parametersgoodfire.ai
- Multimodal Neurons in Artificial Neural Networksdistill.pub
- Information theory in neuroscience - PMCncbi.nlm.nih.gov
- A Geometric Calculator Inside a Neural Networkgoodfire.ai