Google AI Blog: Exploring Transfer Learning with T5: the Text-To-Text Transfer Transformer

Posted by Adam Roberts, Staff Software Engineer and Colin Raffel, Senior Research Scientist, Google Research Over the past few years, tr...

Exploring Transfer Learning with T5: the Text-To-Text Transfer Transformer Skip to main content Exploring Transfer Learning with T5: the Text-To-Text Transfer Transformer February 24, 2020 Posted by Adam Roberts, Staff Software Engineer and Colin Raffel, Senior Research Scientist, Google Research Quick links Share Copy link × Over the past few years, transfer learning has led to a new wave of state-of-the-art results in natural language processing (NLP). Transfer learning's effectiveness comes from pre-training a model on abundantly-available unlabeled text data with a self-supervised task, su

saved by

• Eric Chang

related reading

• TOD-Bertarxiv.org
• GPT-4openai.com
• Transformer Explainer: LLM Transformer Model Visually Explainedpoloclub.github.io
• Generalized Language Models | Lil'Loglilianweng.github.io
• gpt-4.pdfcdn.openai.com
• Transfer Learninglena-voita.github.io
• The Illustrated BERT, ELMo, and co. (How NLP Cracked Transfer Learning) – Jay Alammar – Visualizing machine learning one concept at a time.jalammar.github.io
• The Illustrated BERT, ELMo, and co. (How NLP Cracked Transfer Learning) – Jay Alammar – Visualizing machine learning one concept at a time.jalammar.github.io
• [2111.10952] ExT5: Towards Extreme Multi-Task Scaling for Transfer Learningarxiv.org
• Large Language Models Reading List | Sebastian Raschka, PhDsebastianraschka.com
• Transformers, what can they do? · Hugging Facehuggingface.co
• Recent Advances in Language Model Fine-tuningruder.io