Transfer Learning

Introduction to Transfer Learning and Pretrained Models (ELMo, BERT, GPT).

Transfer Learning p { text-align: justify; } ⇤ NLP Course | For You Transfer Learning What is Transfer Learning? Recap: Word Embeddings Pretrained Models • Words ↦ Words in Context • CoVe: Context Vectors • ELMo: Embeddings from LM • Task-Specific Model ↦ Unified • GPT: Generative Pretraining • BERT: Bidirectional Encoder Benchmarks --> (A Bit of) Adapters (A Note on) Benchmarks Analysis and Interpretability Research Thinking --> Related Papers --> Have Fun! ☰ --> --> (Introduction to) Transfer Learning Lena : Transfer Learning is huge: therefore, it is not

saved by

• Fernando Silva

related reading

• SenseBERT: Driving Some Sense into BERT - ACL Anthologyaclanthology.org
• Seq2seq and Attentionlena-voita.github.io
• The Illustrated BERT, ELMo, and co. (How NLP Cracked Transfer Learning) – Jay Alammar – Visualizing machine learning one concept at a time.jalammar.github.io
• Generalized Language Models | Lil'Loglilianweng.github.io
• The Illustrated BERT, ELMo, and co. (How NLP Cracked Transfer Learning) – Jay Alammar – Visualizing machine learning one concept at a time.jalammar.github.io
• 1810.04805arxiv.org
• [2005.14165] Language Models are Few-Shot Learnersarxiv.org
• Recent Advances in Language Model Fine-tuningruder.io
• radford2018improving.pdfcs.ubc.ca
• Getting Caught Up to Modern LLM Research | Samarth Goeldev.samarthgoel.com
• NLP's ImageNet moment has arrivedthegradient.pub
• 1910.10683arxiv.org