google-research/tuning_playbook: A playbook for systematically maximizing the performance of deep learning models.

A playbook for systematically maximizing the performance of deep learning models.

This is not an officially supported Google product. Varun Godbole†, George E. Dahl†, Justin Gilmer†, Christopher J. Shallue‡, Zachary Nado† † Google Research, Brain Team ‡ Harvard University Table of Contents Who is this document for? Why a tuning playbook? Guide for starting a new project Choosing the model architecture Choosing the optimizer Choosing the batch size Choosing the initial configuration A scientific approach to improving model performance The incremental tuning strategy Exploration vs exploitation Choosing the goal for the next round of experiments Designing…

saved by

• Tazik Sh
• Mehul Raheja
• Jacob G-W
• Ishan Mukherjee

related reading

• GitHub - google-research/tuning_playbook: A playbook for systematically maximizing the performance of deep learning models. · GitHubgithub.com
• A Recipe for Training Neural Networkskarpathy.github.io
• How To Scale Your Modeljax-ml.github.io
• Making Deep Learning go Brrrr From First Principleshorace.io
• The Practitioner’s Guide to the Maximal Update Parameterization - Cerebrascerebras.ai
• Debugging Reinforcement Learning Systemsandyljones.com
• The Practitioner's Guide to the Maximal Update Parameterization | EleutherAI Blogblog.eleuther.ai
• The Little Book of Deep Learningfleuret.org
• A Recipe for Training Neural Networkskarpathy.github.io
• Hyperparameter optimization - Wikipediaen.wikipedia.org
• The Annotated Transformernlp.seas.harvard.edu
• Tiny TPUtinytpu.com