Gradient Routing: Masking Gradients to Localize Computation in Neural Networks — LessWrong

We present gradient routing, a way of controlling where learning happens in neural networks. Gradient routing applies masks to limit the flow of gradients during backpropagation. By supplying different masks for different data points, the user can induce specialized subcomponents within a model. We think gradient routing has the potential to train safer AI systems, for example, by making them more transparent, or by enabling the removal or monitoring of sensitive capabilities. In this post, we: If you’re interested in further discussion or details, check out the paper and its extensive appendices, or the code for gradient routing. Gradient routing allows the user to configure what data (at the level of tokens, documents, or any other feature of the data) causes learning updates where in a neural network (parameters, activations, modules). In full generality, this configuration is achieved by assigning weights to every edge in the computational graph, for every data point. These weights

x Gradient Routing: Masking Gradients to Localize Computation in Neural Networks — LessWrong Best of LessWrong 2024 Interpretability (ML & AI) Machine Unlearning MATS Program Shard Theory AI Frontpage 180 Gradient Routing: Masking Gradients to Localize Computation in Neural Networks by cloud , Jacob G-W , Evzen , Joseph Miller , TurnTrout 6th Dec 2024 AI Alignment Forum Linkpost for arxiv.org 13 min read 16 180 Ω 68 We present gradient routing, a way of controlling where learning happens in neural networks. Gradient routing applies masks to limit the flow of gradients during backpropagation. B

related reading

• Gradient Routing: Masking Gradients to Localize Computation in Neural Networks — AI Alignment Forumalignmentforum.org
• Selective modularity: a research agenda — LessWronglesswrong.com
• LoRA Without Regret - Thinking Machines Labthinkingmachines.ai
• Bitter Lessons from Distillation Robustifies Unlearningbrucewlee.com
• Circuit Tracing: Revealing Computational Graphs in Language Modelstransformer-circuits.pub
• Modular Pretraining Enables Access Controlalignment.anthropic.com
• Andrej Karpathy — AGI is still a decade awaydwarkesh.com
• Attribution Patching: Activation Patching At Industrial Scale - Neel Nandaneelnanda.io
• State of RL for reasoning LLMs | A. Weersaweers.de
• NL.pdfabehrouz.github.io
• The Little Book of Deep Learningfleuret.org
• frontier model training methodologies | Alex Wa's Blogdjdumpling.github.io