A ten-minute introduction to sequence-to-sequence learning in Keras
I see this question a lot -- how to implement RNN sequence-to-sequence learning in Keras? Here is a short introduction. Note that this post assumes that you already have some experience with recurrent networks and Keras. Sequence-to-sequence learning (Seq2Seq) is about training models to convert sequences from one domain (e.g. sentences in English) to sequences in another domain (e.g. the same sentences translated to French). This can be used for machine translation or for free-from question answering (generating a natural language answer given a natural language question) -- in general, it is applicable any time you need to generate text. There are multiple ways to handle this task, either using RNNs or using 1D convnets. Here we will focus on RNNs. When both input sequences and output sequences have the same length, you can implement such models simply with a Keras LSTM or GRU layer (or stack thereof). This is the case in this example script that shows how to teach a RNN to learn to
Note: this post is from 2017. See this tutorial for an up-to-date version of the code used here. I see this question a lot -- how to implement RNN sequence-to-sequence learning in Keras? Here is a short introduction. Note that this post assumes that you already have some experience with recurrent networks and Keras. What is sequence-to-sequence learning? Sequence-to-sequence learning (Seq2Seq) is about training models to convert sequences from one domain (e.g. sentences in English) to sequences in another domain (e.g. the same sentences translated to French). "the cat sat on the mat" -> [ Seq2
Explore this link on the map →saved by
related reading
- Encoder-Decoder Seq2Seq Models, Clearly Explained!! | by Kriz Moses | Analytics Vidhya | Mediummedium.com
- Visualizing A Neural Machine Translation Model (Mechanics of Seq2seq Models With Attention) – Jay Alammar – Visualizing machine learning one concept at a time.jalammar.github.io
- The Unreasonable Effectiveness of Recurrent Neural Networkskarpathy.github.io
- Aman's AI Journal • Primers • Ilya Sutskever's Top 30aman.ai
- Encoder Decoder Models · Hugging Facehuggingface.co
- [1409.3215] Sequence to Sequence Learning with Neural Networksarxiv.org
- transformer_attention.pdfarxiv.org
- Seq2seq and Attentionlena-voita.github.io
- The Illustrated Transformer – Jay Alammar – Visualizing machine learning one concept at a time.jalammar.github.io
- Transformers from Scratche2eml.school
- Recurrent Neural Networks Tutorial, Part 1 – Introduction to RNNs · Denny's Blogdennybritz.com
- Transformers Explained Visually (Part 1): Overview of Functionality | Towards Data Sciencetowardsdatascience.com