⏱️ Performance Profiling - Composer
Introduction: The Composer Profiler enables practitioners to collect, analyze, and visualize performance metrics during training which can be used to diagnose bottlenecks and facilitate model devel...
⏱️ Performance Profiling # Introduction # The Composer Profiler enables practitioners to collect, analyze, and visualize performance metrics during training which can be used to diagnose bottlenecks and facilitate model development. The profiler enables users to capture the following metrics: Duration of each Event, Callback, and Algorithm during training Time taken by the data loader to return a batch Host metrics such as CPU, system memory, disk, and network utilization over time Execution order, latency, and attributes of PyTorch operators and GPU kernels (see torch.profiler ) This tutorial
saved by
related reading
- 🥽 Analyzing Tracesdocs.mosaicml.com
- 👨👩👧👦 Distributed Training - Composerdocs.mosaicml.com
- ⏯️ Autoresume Training - Composerdocs.mosaicml.com
- ⚡ Migrating from PTL - Composerdocs.mosaicml.com
- ♻️ Auto Microbatching - Composerdocs.mosaicml.com
- Mosaic ResNet Deep Dive | Databricks Blogmosaicml.com
- Efficiently Estimating Pareto Frontiers with Cyclic Learning Rate Schedules | Databricks Blogmosaicml.com
- PyTorch Profiling 101 with Modded-NanoGPTblog.underfit.ai
- Composer2.pdfcursor.com
- Making Deep Learning go Brrrr From First Principleshorace.io
- What’s New in PyTorch Profiler 1.9? – PyTorchpytorch.org
- A friendly introduction to machine learning compilers and optimizershuyenchip.com