google-research/tuning_playbook: A playbook for systematically maximizing the performance of deep learning models.

A playbook for systematically maximizing the performance of deep learning models. - google-research/tuning_playbook

Deep Learning Tuning Playbook This is not an officially supported Google product. Varun Godbole † , George E. Dahl † , Justin Gilmer † , Christopher J. Shallue ‡ , Zachary Nado † † Google Research, Brain Team ‡ Harvard University Table of Contents Who is this document for? Why a tuning playbook? Guide for starting a new project Choosing the model architecture Choosing the optimizer Choosing the batch size Choosing the initial configuration A scientific approach to improving model performance The incremental tuning strategy Exploration vs exploitation Choosing the goal for the next round of exp

related reading

• GitHub - google-research/tuning_playbook: A playbook for systematically maximizing the performance of deep learning models. · GitHubgithub.com
• Hyperparameter optimization - Wikipediaen.wikipedia.org
• LoRA Without Regret - Thinking Machines Labthinkingmachines.ai
• The Little Book of Deep Learningfleuret.org
• Scaling Laws For Every Hyperparameter Via Cost-Aware HPO - Imbueimbue.com
• A Recipe for Training Neural Networkskarpathy.github.io
• IsoCompute Playbook: Optimally Scaling Sampling Compute for RL Training of LLMscompute-optimal-rl-llm-scaling.github.io
• A Recipe for Training Neural Networkskarpathy.github.io
• Weights & Biaseswandb.ai
• Evaluation and Fine Tuning | Virgiliovirgili0.github.io
• frontier model training methodologies | Alex Wa's Blogdjdumpling.github.io
• [2604.21691] There Will Be a Scientific Theory of Deep Learningarxiv.org