Backpropagation ≠ Chain Rule – Theory Dish
The chain rule is a fundamental result in calculus. Roughly speaking, it states that if a variable $latex c$ is a differentiable function of intermediate variables $latex b_1,\ldots,b_n$, and each …
The chain rule is a fundamental result in calculus. Roughly speaking, it states that if a variable is a differentiable function of intermediate variables , and each intermediate variable is itself a differentiable function of , then we can compute the derivative as follows: Besides being a handy tool for computing derivatives in calculus homework, the chain rule is closely related to the backpropagation algorithm that is widely-used for computing derivatives (gradients) in neural network training. This blog post by Boaz Barak is a beautiful tutorial on the chain rule and the backpropagation al
Explore this link on the map →saved by
related reading
- Calculus on Computational Graphs: Backpropagation -- colah's blogcolah.github.io
- Backpropagation calculus | 3Blue1Brown3blue1brown.com
- Neural networks and deep learningneuralnetworksanddeeplearning.com
- Backpropagation - Wikipediaen.wikipedia.org
- Backpropagation | Brilliant Math & Science Wikibrilliant.org
- Yes you should understand backprop | by Andrej Karpathy | Mediumkarpathy.medium.com
- CS231n Deep Learning for Computer Visioncs231n.github.io
- What is backpropagation really doing? | 3Blue1Brown3blue1brown.com
- Neural networks and deep learningneuralnetworksanddeeplearning.com
- Differentiable Programming from Scratchthenumb.at
- Coroutines and backpropagation — Eugene Haeugeneha.ca
- The Matrix Calculus You Need For Deep Learningarxiv.org