flâneur

Get Started with Distributed Training using PyTorch Lightning — Ray 2.34.0

docs.ray.io · 2,624 words · saved by 1 readers

This tutorial walks through the process of converting an existing PyTorch Lightning script to use Ray Train. Learn how to: Configure the Lightning Trainer so that it runs distributed with Ray and on the correct CPU or GPU device. Configure training function to report metrics and save checkpoints. Configure scaling and CPU or GPU resource requirements for a training job. Launch a distributed training job with a TorchTrainer. For reference, the final code is as follows: train_func is the Python code that executes on each distributed training worker. ScalingConfig defines the number of distributed training workers and whether to use GPUs. TorchTrainer launches the distributed training job. Compare a PyTorch Lightning training script with and without Ray Train. First, update your training code to support distributed training. Begin by wrapping your code in a training function: Each distributed training worker executes this function. You can also specify the input argument for train_func as

Get Started with Distributed Training using PyTorch Lightning # This tutorial walks through the process of converting an existing PyTorch Lightning script to use Ray Train. Learn how to: Configure the Lightning Trainer so that it runs distributed with Ray and on the correct CPU or GPU device. Configure training function to report metrics and save checkpoints. Configure scaling and CPU or GPU resource requirements for a training job. Launch a distributed training job with a TorchTrainer . Quickstart # For reference, the final code is as follows: from ray.train.torch import TorchTrainer from ray

related reading