Reframing Reinforcement Learning as Sequence Modeling with Transformers?

The Transformer Network, developed by Google and presented in a NeurIPS2017 paper, is one of the few papers that can truly claim to havefundamentally transfo...

The Transformer Network , developed by Google and presented in a NeurIPS 2017 paper , is one of the few papers that can truly claim to have fundamentally transformed (pun intended) the field of Artificial Intelligence. Transformer Networks have become the foundation of some of the most dramatic performance advances in Natural Language Processing (NLP). Two prominent examples are Google’s BERT model , which uses a bidirectional Transformer, and OpenAI’s line of GPT models , which uses a unidirectional Transformer. Both papers have substantially helped out their respective companies’ bottom line

related reading

• A Mathematical Framework for Transformer Circuitstransformer-circuits.pub
• Transformers from Scratche2eml.school
• transformer_attention.pdfarxiv.org
• The Illustrated Transformer – Jay Alammar – Visualizing machine learning one concept at a time.jalammar.github.io
• 2106.01345arxiv.org
• Transformer (deep learning) - Wikipediaen.wikipedia.org
• The Annotated Transformernlp.seas.harvard.edu
• A Introduction to Transformersarxiv.org
• Transformers from Scratchbrandonrohrer.com
• Large Language Models Reading List | Sebastian Raschka, PhDsebastianraschka.com
• Transformer Explainer: LLM Transformer Model Visually Explainedpoloclub.github.io
• The Illustrated Transformer – Jay Alammar – Visualizing machine learning one concept at a time.jalammar.github.io