✳flâneur — a map of the web's best reading

Deep Learning: Our Miraculous Year 1990-1991

people.idsia.ch · 23,097 words · saved by 1 readers

Jürgen Schmidhuber (2019, updated '20, '21, '22, '25, '26) Pronounce: You_again Shmidhoobuh Preprint arxiv:2005.05744 AI Blog @SchmidhuberAI Inaugural tweet: 4 Oct 2019 Abstract. The Deep Learning Artificial Neural Networks (NNs) of our team have revolutionised Machine Learning & AI. Many of the basic ideas behind this revolution were published within the 12 months of our "Annus Mirabilis" 1990-1991 at our lab in TU Munich. Back then, few people were interested. But a quarter century later, NNs based on our "Miraculous Year" were on over 3 billion devices, and used many billions of times per day, consuming a significant fraction of the world's compute. In particular, in 1990-91, we laid foundations of Generative AI, publishing principles of (1) Generative Adversarial Networks for Artificial Curiosity and Creativity (now used for deepfakes), (2) Transformers (the T in ChatGPT—see the 1991 Unnormalized Linear Transformer), (3) Pre-training for deep NNs (see the P in ChatGPT), (4) NN dist

Deep Learning: Our Miraculous Year 1990-1991 -->. Jürgen Schmidhuber ( 2019 , updated '20 , '21 , '22 , '25 , '26) Pronounce: You_again Shmidhoobuh Preprint arxiv:2005.05744 AI Blog @SchmidhuberAI Inaugural tweet : 4 Oct 2019 Deep Learning: Our Miraculous Year 1990-1991 Abstract. The Deep Learning Artificial Neural Networks (NNs) of our team have revolutionised Machine Learning & AI . Many of the basic ideas behind this revolution were published within the 12 months of our "Annus Mirabilis" 1990-1991 at our lab in TU Munich. Back then, few people were interested. But a quarter century lat

Explore this link on the map →

related reading