Long short-term memory

Long short-term memory (LSTM) is an artificial neural network used in the fields of artificial intelligence and deep learning. Unlike standard feedforward neural networks, LSTM has feedback connections. Such a recurrent neural network (RNN) can process not only single data points (such as images), but also entire sequences of data (such as speech or video). This characteristic makes LSTM networks ideal for processing and predicting data. For example, LSTM is applicable to tasks such as unsegmented, connected handwriting recognition, speech recognition, machine translation, speech activity detection, robot control, video games, and healthcare.The name of LSTM refers to the analogy that a standard RNN has both "long-term memory" and "short-term memory". The connection weights and biases in the network change once per episode of training, analogous to how physiological changes in synaptic strengths store long-term memories; the activation patterns in the network change once per time-step, analogous to how the moment-to-moment change in electric firing patterns in the brain store short-term memories. The LSTM architecture aims to provide a short-term memory for RNN that can last thousands of timesteps, thus "long short-term memory".A common LSTM unit is composed of a cell, an input gate, an output gate and a forget gate. The cell remembers values over arbitrary time intervals and the three gates regulate the flow of information into and out of the cell. Forget gates decide what information to discard from a previous state by assigning a previous state, compared to a current input, a value between 0 and 1. A (rounded) value of 1 means to keep the information, and a value of 0 means to discard it. Input gates decide which pieces of new information to store in the current state, using the same system as forget gates. Output gates control which pieces of information in the current state to output by assigning a value from 0 to 1 to the information, considering the previous and current states. Selectively outputting relevant information from the current state allows the LSTM network to maintain useful, long-term dependencies to make predictions, both in current and future time-steps.

Long short-term memory - Wikipedia Jump to content From Wikipedia, the free encyclopedia Recurrent neural network architecture "LSTM" redirects here. For other uses, see LSTM (disambiguation) . This article may be too technical for most readers to understand . Please help improve it to make it understandable to non-experts , without removing the technical details. ( March 2022 ) ( Learn how and when to remove this message ) Part of a series on Machine learning and data mining Paradigms Supervised learning Unsupervised learning Semi-supervised learning Self-supervised learning Reinforcement lea

related reading

• Understanding LSTM Networks -- colah's blogcolah.github.io
• The Unreasonable Effectiveness of Recurrent Neural Networkskarpathy.github.io
• Aman's AI Journal • Primers • Ilya Sutskever's Top 30aman.ai
• Recurrent neural network - Wikipediaen.wikipedia.org
• 遞歸神經網路和長短期記憶模型 RNN & LSTM · 資料科學・機器・人brohrer.mcknote.com
• [2501.00663] Titans: Learning to Memorize at Test Timearxiv.org
• [2606.06479] Pretraining Recurrent Networks without Recurrencearxiv.org
• Recurrent Neural Networks Tutorial, Part 1 – Introduction to RNNs · Denny's Blogdennybritz.com
• NL.pdfabehrouz.github.io
• The Decade of Deep Learning | Leo Gaobmk.sh
• The Unreasonable Effectiveness of Recurrent Neural Networkskarpathy.github.io
• [1410.4615] Learning to Executearxiv.org