Chapter 0: Fundamentals - ARENA
Before we use this model to make any predictions, we first need to think about our input data. Below is a block of code to fetch and process MNIST data. We will go through it line by line. The torchvision package consists of popular datasets, model architectures, and common image transformations for computer vision, and torchvision.transforms provides access to a suite of functions for preprocessing data. We define a transform for the MNIST data (which is applied to each image in the dataset) by composing ToTensor (which converts a PIL.Image object into a PyTorch tensor) and Normalize (which takes arguments for the mean and standard deviation, and performs the linear transformation x -> (x - mean) / std). For the latter, we use 0.1307 and 0.3081 which are the empirical mean & std of the raw data (so after this transformation, the data will have mean 0 and variance 1). Next, we define our datasets using torchvision.datasets. The first argument tells us where to save our data to (so that
2️⃣ Training Neural Networks Learning Objectives Understand how to work with transforms, datasets and dataloaders Understand the basic structure of a training loop Learn how to write your own validation loop Transforms, Datasets & DataLoaders Before we use this model to make any predictions, we first need to think about our input data. Below is a block of code to fetch and process MNIST data. We will go through it line by line. MNIST_TRANSFORM = transforms . Compose ( [ transforms . ToTensor (), transforms . Normalize ( 0.1307 , 0.3081 ), ] ) def get_mnist ( trainset_size : int = 10_000 , test
Explore this link on the map →related reading
- The Little Book of Deep Learningfleuret.org
- A Recipe for Training Neural Networkskarpathy.github.io
- A Recipe for Training Neural Networkskarpathy.github.io
- microgptkarpathy.github.io
- Practical Deep Learning for Coders - Practical Deep Learningcourse.fast.ai
- The Annotated Transformernlp.seas.harvard.edu
- The Annotated Transformernlp.seas.harvard.edu
- Convolutional Neural Network (CNN) | TensorFlow Coretensorflow.org
- frontier model training methodologies | Alex Wa's Blogdjdumpling.github.io
- Transformer Math 101 | EleutherAI Blogblog.eleuther.ai
- Latest | Epoch AIepochai.org
- Fine-tuning a model with the Trainer API · Hugging Facehuggingface.co