Practical Deep Learning for Coders - 3: Neural net foundations

This lesson is based partly on chapter 4 of the book.

Today we’ll be learning about the mathematical foundations of deep learning: Stochastic gradient descent (SGD), and the flexibility of linear functions layered with non-linear activation functions. We’ll be focussing particularly on a popular combination called the Rectified linear function (ReLU). Video This lesson is based partly on chapter 4 of the book. Resources Notebooks for this lesson: HuggingFace Spaces Pets repository Which image models are best? How does a neural net really work? Other resources for the lesson Titanic spreadsheet: see the course repository Titanic data…

saved by

• Michael Dawes

related reading

• Practical Deep Learning for Coders - Practical Deep Learningcourse.fast.ai
• Neural networks and deep learningneuralnetworksanddeeplearning.com
• The Little Book of Deep Learningfleuret.org
• Neural networks and deep learningneuralnetworksanddeeplearning.com
• Understanding Deep Learningudlbook.github.io
• Deep Learningdeeplearningbook.org
• Neural networks and deep learningneuralnetworksanddeeplearning.com
• Neural networks and deep learningneuralnetworksanddeeplearning.com
• [2403.04807] Mathematics of Neural Networks (Lecture Notes Graduate Course)arxiv.org
• nn-notes.pdfboris-hanin.github.io
• Lecture 7 - Deep Learning Foundations: Neural Tangent Kernelsyoutube.com
• Neural networks and deep learningneuralnetworksanddeeplearning.com