Modern Pretraining Strategies: A Hands-On Guide

Finetuning Sessions · Lab 1 / 8

Modern Pretraining Strategies: A Hands-On Guide Finetuning Sessions · Lab 1 / 8 Miguel Otero Pedrido and Antonio Zarauz Moreno Feb 13, 2026 ∙ Paid 40 7 5 Share Welcome to Lab 1 of the Finetuning Sessions ! After a comprehensive, introductory article about Transformers, attention mechanisms, and language model training pipeline, it's time to get our hands dirty and run experiments! 📕 If you haven't read Lesson 1's article, make sure to review it before going forward! The Finetuning Landscape - A Map of Modern LLM Training Miguel Otero Pedrido and Antonio Zarauz Moreno · Feb 11 Read full story

saved by

• Clementine Yang

related reading

• On-Policy Distillation - Thinking Machines Labthinkingmachines.ai
• Let's build GPT: from scratch, in code, spelled out.youtube.com
• Deep Dive into LLMs like ChatGPTyoutube.com
• Recent Advances in Language Model Fine-tuningruder.io
• The Ultimate Guide to Fine-Tuning LLMs from Basics to Breakthroughs: An Exhaustive Review of Technologies, Research, Best Practices, Applied Research Challenges and Opportunities (Version 1.0)arxiv.org
• frontier model training methodologies | Alex Wa's Blogdjdumpling.github.io
• RLHF & Post-Training Course by Nathan Lambertrlhfbook.com
• microgptkarpathy.github.io
• Getting Caught Up to Modern LLM Research | Samarth Goeldev.samarthgoel.com
• GPT in 60 Lines of NumPy | Jay Modyjaykmody.com
• Fine-tuning LLMs Guide | Unsloth Documentationdocs.unsloth.ai
• SFT, RL, and On-Policy Distillation Through a Distributional Lens | whnrehiew.github.io