Reading a Pretraining Run — Chunyuan Deng
charlesdddd.github.io · saved by 1 readers
What to watch in a large pretraining run, in priority order: loss, gradients, tensors, expert balance, residual paths, modality signals, system health, and evaluation.