🥽 Analyzing Traces - Composer
The Composer Profiler relies on PyTorch profiler for model operator profiling and adds additional profiling information for trainer loop events, dataloaders, and system utilization metrics. This do...
Toggle table of contents sidebar The Composer Profiler relies on PyTorch profiler for model operator profiling and adds additional profiling information for trainer loop events, dataloaders, and system utilization metrics. This document highlights additions by the Composer Profiler for augmenting the PyTorch profiling trace with trainer loop events and ways to analyze the collected traces. The profiler records various duration events such as forward (the time for which forward pass of the model ran; event/forward in the images below) and dumps the information in a file using the Chrome…
saved by
related reading
- ⏱️ Performance Profiling - Composerdocs.mosaicml.com
- 👨👩👧👦 Distributed Training - Composerdocs.mosaicml.com
- ⚡ Migrating from PTL - Composerdocs.mosaicml.com
- ⏯️ Autoresume Training - Composerdocs.mosaicml.com
- ♻️ Auto Microbatching - Composerdocs.mosaicml.com
- PyTorch Profiling 101 with Modded-NanoGPTblog.underfit.ai
- Composer2.pdfcursor.com
- Making Deep Learning go Brrrr From First Principleshorace.io
- What’s New in PyTorch Profiler 1.9? – PyTorchpytorch.org
- A friendly introduction to machine learning compilers and optimizershuyenchip.com
- UW PLSE | How does torch.compile work?uwplse.org
- Circuit Tracing: Revealing Computational Graphs in Language Modelstransformer-circuits.pub