Challenges of governing continually learning AI – Runtime Verification
Earlier this month Google Research dropped a new paper titled “Nested Learning,” which introduces a new architecture that they are calling “a new ML paradigm for continual learning”. And it looks l…
Earlier this month Google Research dropped a new paper titled “Nested Learning,” which introduces a new architecture that they are calling “a new ML paradigm for continual learning”. And it looks like a real step towards new ML architectures that can learn and improve over time like humans do. What they’ve essentially done is trained modular neural networks where some of these modules act to immediately process the last tokens, some act as medium-term memory over dozens to hundreds of tokens, and some act as long-term knowledge storage. These networks are able to update themselves at…
saved by
related reading
- How Should We Implement an AI Pause? | Sudarsh Kunnavakkamsudarsh.com
- The Continual Learning Problemjessylin.com
- You can’t imitation-learn how to continual-learn — LessWronglesswrong.com
- Why We Need Continual Learning | Andreessen Horowitza16z.com
- What's so hard about continuous learning?seangoedecke.com
- What are the real problems of continual learning?infinitefaculty.substack.com
- Lil'Loglilianweng.github.io
- NL.pdfabehrouz.github.io
- Trying to actually define continual learningcharlesponeill.com
- Learning Beyond Gradientstrinkle23897.github.io
- Q-learning is not yet scalableseohong.me
- I am worried about near-term non-LLM AI developments — LessWronglesswrong.com