Deep Learning, NLP, and Representations - colah's blog

In the last few years, deep neural networks have dominated pattern recognition. They blew the previous state of the art out of the water for many computer vision tasks. Voice recognition is also moving that way. But despite the results, we have to wonder… why do they work so well? This post reviews some extremely remarkable results in applying deep neural networks to natural language processing (NLP). In doing so, I hope to make accessible one promising answer as to why deep neural networks work. I think it’s a very elegant perspective. A neural network with a hidden layer has universality: given enough hidden units, it can approximate any function. This is a frequently quoted – and even more frequently, misunderstood and applied – theorem. It’s true, essentially, because the hidden layer can be used as a lookup table. For simplicity, let’s consider a perceptron network. A perceptron is a very simple neuron that fires if it exceeds a certain threshold and doesn’t fire if it doesn’t rea

Deep Learning, NLP, and Representations - colah's blog Deep Learning, NLP, and Representations Posted on July 7, 2014 neural networks , deep learning , representations , NLP , recursive neural networks Introduction In the last few years, deep neural networks have dominated pattern recognition. They blew the previous state of the art out of the water for many computer vision tasks. Voice recognition is also moving that way. But despite the results, we have to wonder… why do they work so well? This post reviews some extremely remarkable results in applying deep neural networks to natural languag

related reading

• Neural Networks, Types, and Functional Programming -- colah's blogcolah.github.io
• Aman's AI Journal • Primers • Ilya Sutskever's Top 30aman.ai
• Neural Networks, Manifolds, and Topology -- colah's blogcolah.github.io
• Visualizing Representations: Deep Learning and Human Beings - colah's blogcolah.github.io
• The Decade of Deep Learning | Leo Gaobmk.sh
• The Unreasonable Effectiveness of Recurrent Neural Networkskarpathy.github.io
• Towards Monosemanticity: Decomposing Language Models With Dictionary Learningtransformer-circuits.pub
• The Illustrated Word2vec – Jay Alammar – Visualizing machine learning one concept at a time.jalammar.github.io
• What Is ChatGPT Doing … and Why Does It Work?-Stephen Wolfram Writingswritings.stephenwolfram.com
• Neural networks and deep learningneuralnetworksanddeeplearning.com
• The Illustrated Word2vec – Jay Alammar – Visualizing machine learning one concept at a time.jalammar.github.io
• Glossary of Deep Learning: Word Embedding | by Jaron Collis | Deeper Learning | Mediummedium.com