Recurrent Neural Networks Tutorial, Part 1 – Introduction to RNNs · Denny's Blog

Recurrent Neural Networks (RNNs) are popular models that have shown great promise in many NLP tasks.

Recurrent Neural Networks (RNNs) are popular models that have shown great promise in many NLP tasks. But despite their recent popularity I’ve only found a limited number of resources that throughly explain how RNNs work, and how to implement them. That’s what this tutorial is about. It’s a multi-part series in which I’m planning to cover the following: Introduction to RNNs (this post) Implementing a RNN using Python and Theano Understanding the Backpropagation Through Time (BPTT) algorithm and the vanishing gradient problem Implementing a GRU/LSTM RNN As part of the tutorial we will implement

saved by

• Fernando Silva
• Thi Vũ

related reading

• The Unreasonable Effectiveness of Recurrent Neural Networkskarpathy.github.io
• Understanding LSTM Networks -- colah's blogcolah.github.io
• https://aclanthology.org/P19-1285.pdfaclanthology.org
• Language Modelinglena-voita.github.io
• The Unreasonable Effectiveness of Recurrent Neural Networkskarpathy.github.io
• Recurrent neural network - Wikipediaen.wikipedia.org
• Sequence Modeling with CTCdistill.pub
• Aman's AI Journal • Primers • Ilya Sutskever's Top 30aman.ai
• Seq2seq and Attentionlena-voita.github.io
• [2606.06479] Pretraining Recurrent Networks without Recurrencearxiv.org
• Visualizing A Neural Machine Translation Model (Mechanics of Seq2seq Models With Attention) – Jay Alammar – Visualizing machine learning one concept at a time.jalammar.github.io
• Ilya 30u30arc.net