Modern Pretraining Strategies: A Hands-On Guide

Finetuning Sessions · Lab 1 / 8

Modern Pretraining Strategies: A Hands-On Guide Finetuning Sessions · Lab 1 / 8 Miguel Otero Pedrido and Antonio Zarauz Moreno Feb 13, 2026 ∙ Paid 40 7 5 Share Welcome to Lab 1 of the Finetuning Sessions ! After a comprehensive, introductory article about Transformers, attention mechanisms, and language model training pipeline, it's time to get our hands dirty and run experiments! 📕 If you haven't read Lesson 1's article, make sure to review it before going forward! The Finetuning Landscape - A Map of Modern LLM Training Miguel Otero Pedrido and Antonio Zarauz Moreno · Feb 11 Read full story

saved by

• Clementine Yang

related reading

• frontier model training methodologies | Alex Wa's Blogdjdumpling.github.io
• Recent Advances in Language Model Fine-tuningruder.io
• [2005.14165] Language Models are Few-Shot Learnersarxiv.org
• MAI-Thinking-1: Building a Hill-Climbing Machinemicrosoft.ai
• Anatomy of a Modern Finetuning APIbenanderson.work
• [2509.14786] Pre-training under infinite computearxiv.org
• [2606.07527] Post-training is (Massive) Supervised Learningarxiv.org
• The Ultimate Guide to Fine-Tuning LLMs from Basics to Breakthroughs: An Exhaustive Review of Technologies, Research, Best Practices, Applied Research Challenges and Opportunities (Version 1.0)arxiv.org
• Early Data Exposure Improves Robustness to Subsequent Fine-Tuningarxiv.org
• Self-Adapting Language Modelsarxiv.org
• [2605.12715] Scaling Laws for Mixture Pretraining Under Data Constraintsarxiv.org
• Getting Caught Up to Modern LLM Research | Samarth Goeldev.samarthgoel.com