The Unreasonable Effectiveness of Recurrent Neural Networks
karpathy.github.io · 8,155 words · saved by 19 readers
Musings of a Computer Scientist.
There’s something magical about Recurrent Neural Networks (RNNs). I still remember when I trained my first recurrent network for Image Captioning . Within a few dozen minutes of training my first baby model (with rather arbitrarily-chosen hyperparameters) started to generate very nice looking descriptions of images that were on the edge of making sense. Sometimes the ratio of how simple your model is to the quality of the results you get out of it blows past your expectations, and this was one of those times. What made this result so shocking at the time was that the common wisdom was that RNN
saved by
- Pranav
- Rajan Agarwal
- Claire Wang
- Julian Quevedo
- Ivy Zhang
- Sarah Pan
- Jianmin Chen
- KL Yap
- Laerdon Kim
- Lydia Nottingham
- Yudhister Joel Kumar
- Timothy Kostolansky
related reading
- The Unreasonable Effectiveness of Recurrent Neural Networkskarpathy.github.io
- Understanding LSTM Networks -- colah's blogcolah.github.io
- A Recipe for Training Neural Networkskarpathy.github.io
- Recurrent Neural Networks Tutorial, Part 1 – Introduction to RNNs · Denny's Blogdennybritz.com
- A Recipe for Training Neural Networkskarpathy.github.io
- Aman's AI Journal • Primers • Ilya Sutskever's Top 30aman.ai
- What Is ChatGPT Doing … and Why Does It Work?-Stephen Wolfram Writingswritings.stephenwolfram.com
- Attention and Augmented Recurrent Neural Networksdistill.pub
- Neural Networks, Types, and Functional Programming -- colah's blogcolah.github.io
- Deep Reinforcement Learning: Pong from Pixelskarpathy.github.io
- GitHub - karpathy/char-rnn: Multi-layer Recurrent Neural Networks (LSTM, GRU, RNN) for character-level language models in Torch · GitHubgithub.com
- Learning to (Learn at Test Time): RNNs with Expressive Hidden Statesarxiv.org