Getting Caught Up to Modern LLM Research | Samarth Goel

Samarth Goel's website and portfolio

Getting Caught Up to Modern LLM Research | Samarth Goel Samarth Goel Builder and Entrepreneur © Samarth Goel 2025 Getting Caught Up to Modern LLM Research 2025-01-15 Table of Contents Introduction Preamble Technical Prerequisites Part 0: RNNs, Encoder-Decoder, and Attention Seq2Seq Implementation with RNNs Encoder-Decoder Architecture The Attention Mechanism Extra Resources Part 1: The Transformer Extra Resources Part 2: Past the Original Transformer Early Derivatives Advancing Encoder-Decoder Models Decoding Strategies Scaling up the Generative Pre-trained Transformer Scaling Up [in General]

saved by

• Flora Guo
• Ishaan Panigrahi

related reading

• How LLMs Actually Work | 0xkato0xkato.xyz
• Everything About Transformerskrupadave.com
• GenAI Handbookgenai-handbook.github.io
• Transformer Circuits Threadtransformer-circuits.pub
• Large Language Models Reading List | Sebastian Raschka, PhDsebastianraschka.com
• LLM Resourcesforrestbicker.com
• Understanding Large Language Modelsmagazine.sebastianraschka.com
• Patterns for Building LLM-based Systems & Productseugeneyan.com
• The Llama Hitchiking Guide to Local LLMs – hackerllamaosanseviero.github.io
• Writing an LLM from scratch, part 1 :: Giles' bloggilesthomas.com
• Transformer Explainer: LLM Transformer Model Visually Explainedpoloclub.github.io
• On the Tradeoffs of SSMs and Transformers | Goomba Labgoombalab.github.io