Keynote & Tutorial – Continual Learning and Catastrophic Forgetting
Continual learning is a key aspect of intelligence. The human brain is able to incrementally learn new skills without compromising those that were learned before, as well as to integrate and contrast new information with earlier acquired knowledge. Intriguingly, deep neural networks, although rivaling human intelligence in other ways, almost completely lack this ability to learn continually. Most strikingly, when these networks learn something new, they tend to “catastrophically” forget what was learned before. In recent years, continual learning has become a hot topic in deep learning, and it is considered one of the main open challenges in the field. In this keynote and tutorial, we will provide an overview of the deep learning research on continual learning of the past years. We will do so with a focus on the insights and intuitions that this line of research has generated with regards to the computational principles of continual learning. We hope that these insights will inform and
Keynote & Tutorial – Continual Learning and Catastrophic Forgetting Skip to main content Keynote & Tutorial Wednesday, August 7, 3:20 - 4:00 pm, Kresge Hall (Keynote) Wednesday, August 7, 4:30 - 6:15 pm, Little Kresge (Tutorial) Continual Learning and Catastrophic Forgetting Gido van de Ven 1 , Dhireesha Kudithipudi 2 , 1 KU Leuven, 2 The University of Texas at San Antonio. Continual learning is a key aspect of intelligence. The human brain is able to incrementally learn new skills without compromising those that were learned before, as well as to integrate and contrast new information w
Explore this link on the map →related reading
- Why We Need Continual Learning | Andreessen Horowitza16z.com
- Loss of plasticity in deep continual learning | Naturenature.com
- augustus odena on X: "I have a bunch of thoughts about continual learning and nothing to do with them (I'm working on something else) so I figured I'd just turn them into a post: First: I think people use "continual learning" to point at a cluster of issues that are related but distinct. I'll list" / Xx.com
- The Continual Learning Problemjessylin.com
- NL.pdfabehrouz.github.io
- What are the real problems of continual learning?infinitefaculty.substack.com
- [2302.00487] A Comprehensive Survey of Continual Learning: Theory, Method and Applicationarxiv.org
- Andrej Karpathy — AGI is still a decade awaydwarkesh.com
- How Might We Learn?andymatuschak.org
- Building machines that learn and think like people | Behavioral and Brain Sciences | Cambridge Corecambridge.org
- A Theory of Deep Learning | Elements of a Vector Spaceelonlit.com
- [2401.08623] Wake-Sleep Consolidated Learningarxiv.org