Label Smoothing Explained using Microsoft Excel
In this blogpost, together, we: - Read and understand about Label Smoothing from Rethinking the Inception Architecture for Computer Vision research paper - Look at why we need Label Smoothing? - Re-implement Label Smoothing in Microsoft Excel step by step - Compare the results from our MS Excel implementation with Fastai/PyTorch versions of Label Smoothing Why are we using Microsoft Excel? It’s a valid question you might ask and I wasn’t a big fan of MS Excel either until I saw this video by Jeremy Howard about Cross Entropy Loss. In the video Jeremy explains Cross Entropy Loss using Microsoft Excel. It clicked and I understood it very well even with the fancy math in the cross entropy loss formula. And that is my hope here too! In this blogpost I hope that together we can see past the math and get the intuition for Label Smoothing and then later be able to implement it in a language/framework of our choice. So, let’s get started! Let’s consider we are faced with a multi-class image cl
Label Smoothing Explained using Microsoft Excel 1 Introduction In this blogpost, together, we: - Read and understand about Label Smoothing from Rethinking the Inception Architecture for Computer Vision research paper - Look at why we need Label Smoothing? - Re-implement Label Smoothing in Microsoft Excel step by step - Compare the results from our MS Excel implementation with Fastai / PyTorch versions of Label Smoothing Why are we using Microsoft Excel? It’s a valid question you might ask and I wasn’t a big fan of MS Excel either until I saw this video by Jeremy Howard about Cross Entropy Loss
Explore this link on the map →saved by
related reading
- RLHF: Reinforcement Learning from Human Feedbackhuyenchip.com
- 使用 TensorFlow 學習 Softmax 回歸 (Softmax Regression) | by Leo Chiu | 手寫筆記 | Mediummedium.com
- CS231n Deep Learning for Computer Visioncs231n.github.io
- The Little Book of Deep Learningfleuret.org
- Learning with not Enough Data Part 1: Semi-Supervised Learning | Lil'Loglilianweng.github.io
- Cut Cross Entropy from first principles - Omkaar Kamathomkaark.com
- Distillation Walkthroughvladfeinberg.com
- Thinking about High-Quality Human Data | Lil'Loglilianweng.github.io
- A Gentle Introduction to Cross-Entropy for Machine Learning - MachineLearningMastery.commachinelearningmastery.com
- Additive smoothing - Wikipediaen.wikipedia.org
- Language Modelinglena-voita.github.io
- MAI-Thinking-1: Building a Hill-Climbing Machinemicrosoft.ai