The Illustrated BERT, ELMo, and co. (How NLP Cracked Transfer Learning) – Jay Alammar – Visualizing machine learning one concept at a time.
Discussions: Hacker News (98 points, 19 comments), Reddit r/MachineLearning (164 points, 20 comments) Translations: Chinese (Simplified), French 1, French 2, Japanese, Korean, Persian, Russian, Spanish 2021 Update: I created this brief and highly accessible video intro to BERT The year 2018 has been an inflection point for machine learning models handling text (or more accurately, Natural Language Processing or NLP for short). Our conceptual understanding of how best to represent words and sentences in a way that best captures underlying meanings and relationships is rapidly evolving. Moreover, the NLP community has been putting forward incredibly powerful components that you can freely download and use in your own models and pipelines (It’s been referred to as NLP’s ImageNet moment, referencing how years ago similar developments accelerated the development of machine learning in Computer Vision tasks).
The Illustrated BERT, ELMo, and co. (How NLP Cracked Transfer Learning) Discussions: Hacker News (98 points, 19 comments) , Reddit r/MachineLearning (164 points, 20 comments) Translations: Chinese (Simplified) , French 1 , French 2 , Japanese , Korean , Persian , Russian , Spanish 2021 Update: I created this brief and highly accessible video intro to BERT The year 2018 has been an inflection point for machine learning models handling text (or more accurately, Natural Language Processing or NLP for short). Our conceptual understanding of how best to represent words and sentences in a way that b
saved by
related reading
- The Illustrated BERT, ELMo, and co. (How NLP Cracked Transfer Learning) – Jay Alammar – Visualizing machine learning one concept at a time.jalammar.github.io
- Transfer Learninglena-voita.github.io
- Transformer (deep learning) - Wikipediaen.wikipedia.org
- Generalized Language Models | Lil'Loglilianweng.github.io
- Exploring Transfer Learning with T5: the Text-To-Text Transfer Transformerai.googleblog.com
- The Illustrated Transformer – Jay Alammar – Visualizing machine learning one concept at a time.jalammar.github.io
- 1810.04805arxiv.org
- Recent Advances in Language Model Fine-tuningruder.io
- NLP's ImageNet moment has arrivedthegradient.pub
- radford2018improving.pdfcs.ubc.ca
- 1910.10683arxiv.org
- A History of Large Language Modelsgregorygundersen.com