Metalearning or Learning to Learn Since 1987

Jürgen Schmidhuber (Dec 2020, 2022, 2024, 2025) Pronounce: You_again Shmidhoobuh AI Blog Twitter: @SchmidhuberAI Abstract. In 1987, I published my first paper on metalearning or learning to learn: my diploma thesis [META1] (Sec. 1). For its cover I drew a robot that bootstraps itself (image above). [META1] was the first in a long series of publications on this topic, which became hot in the 2010s [DEC]. Here I also summarize our work on meta-reinforcement learning with self-modifying policies since 1994 [METARL2-9] (Sec. 2), gradient descent-based metalearning in artificial neural networks since 1992 [FWPMETA1-5] (Sec. 3), asymptotically optimal metalearning for curriculum learning since 2002 [OOPS1-3] (Sec. 4), mathematically optimal metalearning through the self-referential Gödel Machine since 2003 [GM3-9] (Sec. 5), Meta-RL combined with artificial curiosity and intrinsic motivation [AC] since 1990/1997 (Sec. 6), and recent work of 2020-2022 (Sec. 7). The "in-context learning" of rec

Metalearning or Learning to Learn Since 1987 -->. Jürgen Schmidhuber (Dec 2020, 2022 , 2024 , 2025) Pronounce: You_again Shmidhoobuh AI Blog Twitter: @SchmidhuberAI Metalearning Machines Learn to Learn (1987-) Abstract. In 1987, I published my first paper on metalearning or learning to learn : my diploma thesis [META1] ( Sec. 1 ). For its cover I drew a robot that bootstraps itself (image above). [META1] was the first in a long series of publications on this topic, which became hot in the 2010s [DEC] . Here I also summarize our work on meta-reinforcement learning with self-modifying polic

saved by

• Lydia Nottingham

related reading

• Juergen Schmidhuber's home page - Universal Artificial Intelligence - AI - Deep Learning - Recurrent Neural Networks - Computer Vision - Object Detection - Image segmentation - GANs - Transformers with linearized self-attention - Goedel Macpeople.idsia.ch
• Selected Publicationsjoschu.net
• [2212.07677] Transformers learn in-context by gradient descentarxiv.org
• The Scaling Hypothesis · Gwern.netgwern.net
• The Decade of Deep Learning | Leo Gaobmk.sh
• Deep Learning: Our Miraculous Year 1990-1991people.idsia.ch
• Andrej Karpathy — AGI is still a decade awaydwarkesh.com
• Just Ask for Generalization | Eric Jangevjang.com
• NL.pdfabehrouz.github.io
• GenAI Handbookgenai-handbook.github.io
• Why We Need Continual Learning | Andreessen Horowitza16z.com
• Meta Learninglilianweng.github.io