Transfer Learning

Introduction to Transfer Learning and Pretrained Models (ELMo, BERT, GPT).

Transfer Learning p { text-align: justify; } ⇤ NLP Course | For You Transfer Learning What is Transfer Learning? Recap: Word Embeddings Pretrained Models • Words ↦ Words in Context • CoVe: Context Vectors • ELMo: Embeddings from LM • Task-Specific Model ↦ Unified • GPT: Generative Pretraining • BERT: Bidirectional Encoder Benchmarks --> (A Bit of) Adapters (A Note on) Benchmarks Analysis and Interpretability Research Thinking --> Related Papers --> Have Fun! ☰ --> --> (Introduction to) Transfer Learning Lena : Transfer Learning is huge: therefore, it is not

saved by

• Fernando Silva

related reading

• Seq2seq and Attentionlena-voita.github.io
• Language Modelinglena-voita.github.io
• The Illustrated BERT, ELMo, and co. (How NLP Cracked Transfer Learning) – Jay Alammar – Visualizing machine learning one concept at a time.jalammar.github.io
• The Illustrated Transformer – Jay Alammar – Visualizing machine learning one concept at a time.jalammar.github.io
• The Illustrated BERT, ELMo, and co. (How NLP Cracked Transfer Learning) – Jay Alammar – Visualizing machine learning one concept at a time.jalammar.github.io
• Generalized Language Models | Lil'Loglilianweng.github.io
• Transformers from Scratche2eml.school
• transformer_attention.pdfarxiv.org
• Recent Advances in Language Model Fine-tuningruder.io
• 1810.04805arxiv.org
• Exploring Transfer Learning with T5: the Text-To-Text Transfer Transformerai.googleblog.com
• NLP's ImageNet moment has arrivedthegradient.pub