Ambiguous out-of-distribution generalization on an algorithmic task — LessWrong

Introduction It's now well known that simple neural network models often "grok" algorithmic tasks. That is, when trained for many epochs on a subset…

x Ambiguous out-of-distribution generalization on an algorithmic task — LessWrong Deceptive Alignment Distributional Shifts Grokking (ML) Interpretability (ML & AI) MATS Program AI Frontpage 84 Ambiguous out-of-distribution generalization on an algorithmic task by Wilson Wu , Louis Jaburi 13th Feb 2025 13 min read 6 84 Introduction It's now well known that simple neural network models often "grok" algorithmic tasks. That is, when trained for many epochs on a subset of the full input space, the model quickly attains perfect train accuracy and then, much later, near-perfect test accuracy. In the

related reading

• Do Machine Learning Models Memorize or Generalize?pair.withgoogle.com
• Grokking: Generalization Beyond Overfitting on Small Algorithmic Datasetsmathai-iclr.github.io
• Clare Lyle | What's grokking good for?clarelyle.com
• Just Ask for Generalization | Eric Jangevjang.com
• Zipfian grokking | Jasper Gilleyjagilley.github.io
• arxiv.org/pdf/1805.08522arxiv.org
• Understanding deep learning requires rethinking generalizationarxiv.org
• The generalization phase diagram — LessWronglesswrong.com
• AlgZoo: uninterpreted models with fewer than 1,500 parameters — LessWronglesswrong.com
• Narrow Misalignment is Hard, Emergent Misalignment is Easy — LessWronglesswrong.com
• A Mechanistic Interpretability Analysis of Grokking — LessWronglesswrong.com
• [2605.01172] A Theory of Generalization in Deep Learningarxiv.org