Coroutines and backpropagation — Eugene Ha

The backpropagation algorithm for feedforward neural networks is an immediate corollary of the backpropagation equations. We show that an implementation of the algorithm is itself an immediate corollary, provided that we interpret the backpropagation equations as a coroutine.

2024 March 29 Coroutines and backpropagation \( \def\∂#1/∂#2{\mathchoice {\frac{\partial #1}{\partial #2}} {{\partial #1}/{\partial #2}} {{\partial #1}/{\partial #2}} {{\partial #1}/{\partial #2}}} \newcommand\optensor\otimes \newcommand\tensor{{\mskip2mu}{\optensor}{\mskip2mu}} \newcommand\ophadamard\odot \newcommand\hadamard{{\mskip2mu}{\ophadamard}{\mskip2mu}} \newcommand\vec[1]{\mathbf{#1}} % vectorisation \newcommand\upsigma{\unicode{963}} % upright 'σ' \newcommand\act{\vec\upsigma} % activation function \newcommand\sact\sigma % scalar activation \newcommand\Dact{\vec{D\upsigma}} \newcomm

related reading

• Yes you should understand backprop | by Andrej Karpathy | Mediumkarpathy.medium.com
• Neural networks and deep learningneuralnetworksanddeeplearning.com
• Calculus on Computational Graphs: Backpropagation -- colah's blogcolah.github.io
• Neural networks and deep learningneuralnetworksanddeeplearning.com
• A Recipe for Training Neural Networkskarpathy.github.io
• Understanding the Neural Tangent Kernel – EigenTaleseigentales.com
• nn-notes.pdfboris-hanin.github.io
• Backpropagation - Wikipediaen.wikipedia.org
• Some Math behind Neural Tangent Kernel | Lil'Loglilianweng.github.io
• Backpropagation ≠ Chain Rule – Theory Dishtheorydish.blog
• Backpropagation | Brilliant Math & Science Wikibrilliant.org
• A Recipe for Training Neural Networkskarpathy.github.io