Label Smoothing Explained using Microsoft Excel

In this blogpost, together, we: - Read and understand about Label Smoothing from Rethinking the Inception Architecture for Computer Vision research paper - Look at why we need Label Smoothing? - Re-implement Label Smoothing in Microsoft Excel step by step - Compare the results from our MS Excel implementation with Fastai/PyTorch versions of Label Smoothing Why are we using Microsoft Excel? It’s a valid question you might ask and I wasn’t a big fan of MS Excel either until I saw this video by Jeremy Howard about Cross Entropy Loss. In the video Jeremy explains Cross Entropy Loss using Microsoft Excel. It clicked and I understood it very well even with the fancy math in the cross entropy loss formula. And that is my hope here too! In this blogpost I hope that together we can see past the math and get the intuition for Label Smoothing and then later be able to implement it in a language/framework of our choice. So, let’s get started! Let’s consider we are faced with a multi-class image cl

Label Smoothing Explained using Microsoft Excel 1 Introduction In this blogpost, together, we: - Read and understand about Label Smoothing from Rethinking the Inception Architecture for Computer Vision research paper - Look at why we need Label Smoothing? - Re-implement Label Smoothing in Microsoft Excel step by step - Compare the results from our MS Excel implementation with Fastai / PyTorch versions of Label Smoothing Why are we using Microsoft Excel? It’s a valid question you might ask and I wasn’t a big fan of MS Excel either until I saw this video by Jeremy Howard about Cross Entropy Loss

saved by

• Nick Dupoux

related reading

• Things that confused me about cross-entropychris-said.io
• A Recipe for Training Neural Networkskarpathy.github.io
• interpreting GPT: the logit lens — LessWronglesswrong.com
• Distillation Walkthroughvladfeinberg.com
• Understanding softmax and the negative log-likelihoodljvmiranda921.github.io
• Your Softmax Is Broken. Here's the Fix.youtube.com
• Transformers Laid Outgoyalpramod.github.io
• A Gentle Introduction to Cross-Entropy for Machine Learning - MachineLearningMastery.commachinelearningmastery.com
• Cut Cross Entropy from first principles - Omkaar Kamathomkaark.com
• A Recipe for Training Neural Networkskarpathy.github.io
• CS231n Deep Learning for Computer Visioncs231n.github.io
• Understanding and Coding Self-Attention, Multi-Head Attention, Causal-Attention, and Cross-Attention in LLMsmagazine.sebastianraschka.com