contents | Build a Large Language Model (From Scratch)
Explore our new AI Academy tracks and browse curated GenAI content. preface acknowledgments about this book about the author about the cover illustration 1 Understanding large language models 1.1 What is an LLM? 1.2 Applications of LLMs 1.3 Stages of building and using LLMs 1.4 Introducing the transformer architecture 1.5 Utilizing large datasets 1.6 A closer look at the GPT architecture 1.7 Building a large language model 2 Working with text data 2.1 Understanding word embeddings 2.2 Tokenizing text 2.3 Converting tokens into token IDs 2.4 Adding special context tokens 2.5 Byte pair encoding 2.6 Data sampling with a sliding window 2.7 Creating token embeddings 2.8 Encoding word positions 3 Coding attention mechanisms 3.1 The problem with modeling long sequences 3.2 Capturing data dependencies with attention mechanisms 3.3 Attending to different parts of the input with self-attention 3.3.1 A simple self-attention mechanism without trainable weights 3.3.2 Computing attention weights for
Explore this link on the map →