karpathy/char-rnn: Multi-layer Recurrent Neural Networks (LSTM, GRU, RNN) for character-level language models in Torch
Multi-layer Recurrent Neural Networks (LSTM, GRU, RNN) for character-level language models in Torch This code implements multi-layer Recurrent Neural Network (RNN, LSTM, and GRU) for training/sampling from character-level language models. In other words the model takes one text file as input and trains a Recurrent Neural Network that learns to predict the next character in a sequence. The RNN can then be used to generate text character by character that will look like the original training data. The context of this code base is described in detail in my blog post. If you are new to Torch/Lua/Neural Nets, it might be helpful to know that this code is really just a slightly more fancy version of this 100-line gist that I wrote in Python/numpy. The code in this repo additionally: allows for multiple layers, uses an LSTM instead of a vanilla RNN, has more supporting code for model checkpointing, and is of course much more efficient since it uses mini-batches and can run on a GPU. Justin Jo
char-rnn This code implements multi-layer Recurrent Neural Network (RNN, LSTM, and GRU) for training/sampling from character-level language models. In other words the model takes one text file as input and trains a Recurrent Neural Network that learns to predict the next character in a sequence. The RNN can then be used to generate text character by character that will look like the original training data. The context of this code base is described in detail in my blog post . If you are new to Torch/Lua/Neural Nets, it might be helpful to know that this code is really just a slightly more fanc
Explore this link on the map →related reading
- The Unreasonable Effectiveness of Recurrent Neural Networkskarpathy.github.io
- The Unreasonable Effectiveness of Recurrent Neural Networkskarpathy.github.io
- Aman's AI Journal • Primers • Ilya Sutskever's Top 30aman.ai
- Neural Networks: Zero To Herokarpathy.ai
- Language Modelinglena-voita.github.io
- Recurrent Neural Networks Tutorial, Part 1 – Introduction to RNNs · Denny's Blogdennybritz.com
- Recursive Language Models | Alex L. Zhangalexzhang13.github.io
- microgptkarpathy.github.io
- RLHF: Reinforcement Learning from Human Feedbackhuyenchip.com
- frontier model training methodologies | Alex Wa's Blogdjdumpling.github.io
- Understanding LSTM Networks -- colah's blogcolah.github.io
- GenAI Handbookgenai-handbook.github.io