Ambiguous out-of-distribution generalization on an algorithmic task — LessWrong

Introduction It's now well known that simple neural network models often "grok" algorithmic tasks. That is, when trained for many epochs on a subset…

x Ambiguous out-of-distribution generalization on an algorithmic task — LessWrong Deceptive Alignment Distributional Shifts Grokking (ML) Interpretability (ML & AI) MATS Program AI Frontpage 84 Ambiguous out-of-distribution generalization on an algorithmic task by Wilson Wu , Louis Jaburi 13th Feb 2025 13 min read 6 84 Introduction It's now well known that simple neural network models often "grok" algorithmic tasks. That is, when trained for many epochs on a subset of the full input space, the model quickly attains perfect train accuracy and then, much later, near-perfect test accuracy. In the

related reading

• Do Machine Learning Models Memorize or Generalize?pair.withgoogle.com
• [2201.02177] Grokking: Generalization Beyond Overfitting on Small Algorithmic Datasetsarxiv.org
• Grokking: Generalization Beyond Overfitting on Small Algorithmic Datasetsmathai-iclr.github.io
• [2605.01172] A Theory of Generalization in Deep Learningarxiv.org
• Zipfian grokking | Jasper Gilleyjagilley.github.io
• AlgZoo: uninterpreted models with fewer than 1,500 parameters — LessWronglesswrong.com
• [1911.08731] Distributionally Robust Neural Networks for Group Shifts: On the Importance of Regularization for Worst-Case Generalizationarxiv.org
• Weight-Sparse Circuits May Be Interpretable Yet Unfaithful — LessWronglesswrong.com
• Understanding deep learning requires rethinking generalizationarxiv.org
• How do LLMs generalize when we do training that is intuitively compatible with two off-distribution behaviors? — LessWronglesswrong.com
• The generalization phase diagram — LessWronglesswrong.com
• Investigating the learning coefficient of modular addition: hackathon project — LessWronglesswrong.com