Getting Caught Up to Modern LLM Research | Samarth Goel

Samarth Goel's website and portfolio

Getting Caught Up to Modern LLM Research | Samarth Goel Samarth Goel Builder and Entrepreneur © Samarth Goel 2025 Getting Caught Up to Modern LLM Research 2025-01-15 Table of Contents Introduction Preamble Technical Prerequisites Part 0: RNNs, Encoder-Decoder, and Attention Seq2Seq Implementation with RNNs Encoder-Decoder Architecture The Attention Mechanism Extra Resources Part 1: The Transformer Extra Resources Part 2: Past the Original Transformer Early Derivatives Advancing Encoder-Decoder Models Decoding Strategies Scaling up the Generative Pre-trained Transformer Scaling Up [in General]

saved by

• Flora Guo
• Ali Ahmad
• Ishaan Panigrahi

related reading

• LLM (ML) Job Interviews - Resources • Mimansa Jaiswalmimansajaiswal.github.io
• How LLMs Actually Work | 0xkato0xkato.xyz
• The Big LLM Architecture Comparisonmagazine.sebastianraschka.com
• Everything About Transformerskrupadave.com
• LLM Resourcesforrestbicker.com
• Transformers Laid Outgoyalpramod.github.io
• A History of Large Language Modelsgregorygundersen.com
• A Mathematical Framework for Transformer Circuitstransformer-circuits.pub
• GenAI Handbookgenai-handbook.github.io
• What We’ve Learned From A Year of Building with LLMs – Applied LLMsapplied-llms.org
• Understanding Large Language Modelsmagazine.sebastianraschka.com
• Understanding and Coding Self-Attention, Multi-Head Attention, Causal-Attention, and Cross-Attention in LLMsmagazine.sebastianraschka.com