7.pdf

N/A

Speech and Language Processing. Daniel Jurafsky & James H. Martin. Copyright © 2026. All rights reserved. Draft of August 19, 2026. CHAPTER Transformers and Pretraining 7 “The true art of memory is the art of attention ” Samuel Johnson, Idler #74, September 1759 In this chapter we introduce the transformer, the standard architecture for build- ing large language models, and how to pretrain them and how to use them to generate…

related reading

• The Illustrated Transformer – Jay Alammar – Visualizing machine learning one concept at a time.jalammar.github.io
• transformer_attention.pdfarxiv.org
• A Mathematical Framework for Transformer Circuitstransformer-circuits.pub
• Transformers from Scratche2eml.school
• The Illustrated Transformer – Jay Alammar – Visualizing machine learning one concept at a time.jalammar.github.io
• Everything About Transformerskrupadave.com
• A Conceptual Guide to Transformers: Part Ibenlevinstein.substack.com
• The Annotated Transformernlp.seas.harvard.edu
• A Introduction to Transformersarxiv.org
• Transformer (deep learning) - Wikipediaen.wikipedia.org
• 11.7. The Transformer Architecture — Dive into Deep Learning 1.0.3 documentationd2l.ai
• 1706.03762arxiv.org