Connectionist temporal classification
Connectionist temporal classification (CTC) is a type of neural network output and associated scoring function, for training recurrent neural networks (RNNs) such as LSTM networks to tackle sequence problems where the timing is variable. It can be used for tasks like on-line handwriting recognition or recognizing phonemes in speech audio. CTC refers to the outputs and scoring, and is independent of the underlying neural network structure. It was introduced in 2006.The input is a sequence of observations, and the outputs are a sequence of labels, which can include blank outputs. The difficulty of training comes from there being many more observations than there are labels. For example in speech audio there can be multiple time slices which correspond to a single phoneme. Since we don't know the alignment of the observed sequence with the target labels we predict a probability distribution at each time step. A CTC network has a continuous output (e.g. softmax), which is fitted through training to model the probability of a label. CTC does not attempt to learn boundaries and timings: Label sequences are considered equivalent if they differ only in alignment, ignoring blanks. Equivalent label sequences can occur in many ways – which makes scoring a non-trivial task, but there is an efficient forward–backward algorithm for that.
Connectionist temporal classification - Wikipedia Jump to content From Wikipedia, the free encyclopedia Type of neural network output and associated scoring function Connectionist temporal classification ( CTC ) is a loss function and output representation used to train neural networks on sequence-labelling tasks where the input and output are not aligned in time. It can be used for tasks like on-line handwriting recognition [ 1 ] or speech recognition . CTC refers to the outputs and scoring, and is independent of the underlying neural network structure. It was introduced in 2006. [ 2 ] The in
Explore this link on the map →related reading
- Sequence Modeling with CTCdistill.pub
- The Unreasonable Effectiveness of Recurrent Neural Networkskarpathy.github.io
- Aman's AI Journal • Primers • Ilya Sutskever's Top 30aman.ai
- Continuous Thought Machinespub.sakana.ai
- Audio Deep Learning Made Simple: Automatic Speech Recognition (ASR), How it Works | Towards Data Sciencetowardsdatascience.com
- Long short-term memory - Wikipediaen.wikipedia.org
- [2606.06479] Pretraining Recurrent Networks without Recurrencearxiv.org
- Understanding LSTM Networks -- colah's blogcolah.github.io
- What Is ChatGPT Doing … and Why Does It Work?-Stephen Wolfram Writingswritings.stephenwolfram.com
- Recurrent neural network - Wikipediaen.wikipedia.org
- [1409.3215] Sequence to Sequence Learning with Neural Networksarxiv.org
- Recurrent Neural Networks Tutorial, Part 1 – Introduction to RNNs · Denny's Blogdennybritz.com