MIT 6.S191 (2023): Recurrent Neural Networks, Transformers, and Attention - YouTube
This MIT lecture introduces sequence modeling using recurrent neural networks (RNNs). The class explores RNN intuition and design criteria for sequential data, including variable-length sequences and long-term dependencies. Practical examples and code implementations illustrate RNNs and their limitations, paving the way for more advanced architectures. Follow along using the transcript. This MIT lecture introduces sequence modeling using recurrent neural networks (RNNs). The class explores RNN intuition and design criteria for sequential data, including variable-length sequences and long-term dependencies. Practical examples and code implementations illustrate RNNs and their limitations, paving the way for more advanced architectures. Follow along using the transcript.
MIT Introduction to Deep Learning 6.S191: Lecture 2 Recurrent Neural Networks Lecturer: Ava Amini 2023 Edition For all lectures, slides, and lab materials: http://introtodeeplearning.com Lecture Outline 0:00 - Introduction 3:07 - Sequence modeling 5:09 - Neurons with recurrence 12:05 - Recurrent neural networks 13:47 - RNN intuition 15:03 - Unfolding RNNs 18:57 - RNNs from scratch 21:50 - Design criteria for sequential modeling 23:45 - Word prediction example 29:57 - Backpropagation through time 32:25 - Gradient issues 37:03 - Long short term memory (LSTM) 39:50 - RNN applications…
saved by
related reading
- The Unreasonable Effectiveness of Recurrent Neural Networkskarpathy.github.io
- Aman's AI Journal • Primers • Ilya Sutskever's Top 30aman.ai
- Visualizing A Neural Machine Translation Model (Mechanics of Seq2seq Models With Attention) – Jay Alammar – Visualizing machine learning one concept at a time.jalammar.github.io
- Attention and Augmented Recurrent Neural Networksdistill.pub
- Recurrent Neural Networks Tutorial, Part 1 – Introduction to RNNs · Denny's Blogdennybritz.com
- Understanding LSTM Networks -- colah's blogcolah.github.io
- transformer_attention.pdfarxiv.org
- [2606.06479] Pretraining Recurrent Networks without Recurrencearxiv.org
- Seq2seq and Attentionlena-voita.github.io
- The Annotated Transformernlp.seas.harvard.edu
- 6.S898 Deep Learning, Fall 2023phillipi.github.io
- MIT Deep Learning 6.S191introtodeeplearning.com