Batch size 1 or ∞
kothasuhas.github.io · 1,508 words · saved by 1 readers
Batch size 1 or ∞
10/6/2026 Batch size is important. Like other hyperparameters, it may be unclear what value is optimal for your setting. Unlike other hyperparameters (like learning rate and momentum), batch size has a complicated interaction with the actual resource constraints of the problem. It is common to hear inconsistent intuitions such as "you want the biggest batch size that fits on your hardware", "smaller batch size gets better generalization", and "you want a tuned medium batch size for practice". In fact, all of these statements are true, just under different resource constraints. This post is…
saved by
related reading
- IsoCompute Playbook: Optimally Scaling Sampling Compute for RL Training of LLMscompute-optimal-rl-llm-scaling.github.io
- All About Rooflines | How To Scale Your Modeljax-ml.github.io
- Making Deep Learning go Brrrr From First Principleshorace.io
- Transformer Inference Arithmetic | kipply's blogkipp.ly
- Jane Street Blog - Does batch size matter?blog.janestreet.com
- Scaling Laws, Carefully | Lil'Loglilianweng.github.io
- Infinite Limits of Neural Networks - Kempner Institutekempnerinstitute.harvard.edu
- [2509.14786] Pre-training under infinite computearxiv.org
- Transformer Math 101 | EleutherAI Blogblog.eleuther.ai
- [2601.05034] How to Set the Batch Size for Large-Scale Pre-training?arxiv.org
- Pre-training under infinite computearxiv.org
- Transformers Inference Optimization Toolset | AstraBlogastralord.github.io