Gradient Routing: Masking Gradients to Localize Computation in Neural Networks — AI Alignment Forum

Isolate capabilities to known parts of a neural network. Helps with interpretability, robust unlearning, and scalable oversight.

x Gradient Routing: Masking Gradients to Localize Computation in Neural Networks — AI Alignment Forum Best of LessWrong 2024 Interpretability (ML & AI) Machine Unlearning MATS Program Shard Theory AI Frontpage 68 Gradient Routing: Masking Gradients to Localize Computation in Neural Networks by cloud , Jacob G-W , Evzen , Joseph Miller , TurnTrout 6th Dec 2024 Linkpost for arxiv.org 13 min read 16 68 We present gradient routing, a way of controlling where learning happens in neural networks. Gradient routing applies masks to limit the flow of gradients during backpropagation. By supplying diffe

related reading

• Gradient Routing: Masking Gradients to Localize Computation in Neural Networks — LessWronglesswrong.com
• 2410.04332arxiv.org
• Selective modularity: a research agenda — LessWronglesswrong.com
• Self-Fulfilling Misalignment Data Might Be Poisoning Our AI Modelsturntrout.com
• Recommendations for Technical AI Safety Research Directionsalignment.anthropic.com
• Auditing language models for hidden objectives — LessWronglesswrong.com
• Modular Pretraining Enables Access Controlalignment.anthropic.com
• In (highly contingent!) defense of interpretability-in-the-loop ML training — AI Alignment Forumalignmentforum.org
• [2512.05648] Beyond Data Filtering: Knowledge Localization for Capability Removal in LLMsarxiv.org
• Language Model Circuits Are Sparse in the Neuron Basis | Transluce AItransluce.org
• Alignment Faking Mitigationsalignment.anthropic.com
• Mechanistically Eliciting Latent Behaviors in Language Models — AI Alignment Forumalignmentforum.org