Practical Deep Learning for Coders - 3: Neural net foundations

This lesson is based partly on chapter 4 of the book.

Today we’ll be learning about the mathematical foundations of deep learning: Stochastic gradient descent (SGD), and the flexibility of linear functions layered with non-linear activation functions. We’ll be focussing particularly on a popular combination called the Rectified linear function (ReLU). Video This lesson is based partly on chapter 4 of the book. Resources Notebooks for this lesson: HuggingFace Spaces Pets repository Which image models are best? How does a neural net really work? Other resources for the lesson Titanic spreadsheet: see the course repository Titanic data…

saved by

• Michael Dawes

related reading

• Understanding Deep Learningudlbook.github.io
• Neural networks and deep learningneuralnetworksanddeeplearning.com
• The Little Book of Deep Learningfleuret.org
• [2403.04807] Mathematics of Neural Networks (Lecture Notes Graduate Course)arxiv.org
• Neural networks and deep learningneuralnetworksanddeeplearning.com
• Practical Deep Learning for Coders - Practical Deep Learningcourse.fast.ai
• Deep Learningdeeplearningbook.org
• ReLU Activation Function in Deep Learning - GeeksforGeeksgeeksforgeeks.org
• Neural networks and deep learningneuralnetworksanddeeplearning.com
• Ilya 30u30arc.net
• The Decade of Deep Learning | Leo Gaobmk.sh
• nn-notes.pdfboris-hanin.github.io