Google AI Blog: Exploring Transfer Learning with T5: the Text-To-Text Transfer Transformer

Posted by Adam Roberts, Staff Software Engineer and Colin Raffel, Senior Research Scientist, Google Research Over the past few years, tr...

Exploring Transfer Learning with T5: the Text-To-Text Transfer Transformer Skip to main content Exploring Transfer Learning with T5: the Text-To-Text Transfer Transformer February 24, 2020 Posted by Adam Roberts, Staff Software Engineer and Colin Raffel, Senior Research Scientist, Google Research Quick links Share Copy link × Over the past few years, transfer learning has led to a new wave of state-of-the-art results in natural language processing (NLP). Transfer learning's effectiveness comes from pre-training a model on abundantly-available unlabeled text data with a self-supervised task, su

saved by

• Eric Chang
• Alex Shaw

related reading

• Transformer (deep learning) - Wikipediaen.wikipedia.org
• Recent Advances in Language Model Fine-tuningruder.io
• transformer_attention.pdfarxiv.org
• Transformers Explained Visually (Part 1): Overview of Functionality | Towards Data Sciencetowardsdatascience.com
• The Illustrated BERT, ELMo, and co. (How NLP Cracked Transfer Learning) – Jay Alammar – Visualizing machine learning one concept at a time.jalammar.github.io
• A Mathematical Framework for Transformer Circuitstransformer-circuits.pub
• 1910.10683arxiv.org
• [2005.14165] Language Models are Few-Shot Learnersarxiv.org
• Transfer Learninglena-voita.github.io
• Transformers from Scratche2eml.school
• The Annotated Transformernlp.seas.harvard.edu
• The Annotated Transformernlp.seas.harvard.edu