It's Time to Move On: Primacy Bias and Why It Helps to Forget | ICLR Blogposts 2024

'The Primacy Bias in Deep Reinforcement Learning' demonstrates how the first experiences of a deep learning model can cause catastrophic memorization and how this can be prevented. In this post we describe primacy bias, summarize the authors' key findings, and present a simple environment to experiment with primacy bias.

It's Time to Move On: Primacy Bias and Why It Helps to Forget | ICLR Blogposts 2024 It's Time to Move On: Primacy Bias and Why It Helps to Forget 'The Primacy Bias in Deep Reinforcement Learning' demonstrates how the first experiences of a deep learning model can cause catastrophic memorization and how this can be prevented. In this post we describe primacy bias, summarize the authors' key findings, and present a simple environment to experiment with primacy bias. Introduction to Primacy Bias Primacy bias occurs when a model’s training is damaged by overfitting to its first experiences. This c

related reading

• Just Ask for Generalization | Eric Jangevjang.com
• State of RL for reasoning LLMs | A. Weersaweers.de
• The Little Book of Deep Learningfleuret.org
• NL.pdfabehrouz.github.io
• Andrej Karpathy — AGI is still a decade awaydwarkesh.com
• The Decade of Deep Learning | Leo Gaobmk.sh
• Deep Reinforcement Learning Doesn't Work Yetalexirpan.com
• Deep Q-Networks Explained — LessWronglesswrong.com
• Learning Beyond Gradientstrinkle23897.github.io
• Zipfian grokking | Jasper Gilleyjagilley.github.io
• Understanding Memorization via Loss Curvaturegoodfire.ai
• Key Papers in Deep RL - Spinning Up documentationspinningup.openai.com