✳flâneur — a map of the web's best reading

Reverse-mode automatic differentiation from scratch, in Python

sidsite.com · 4,045 words · saved by 1 readers

The site of Sid.

Automatic differentiation is the foundation upon which deep learning frameworks lie. Deep learning models are typically trained using gradient based techniques, and autodiff makes it easy to get gradients, even from enormous, complex models. ‘Reverse-mode autodiff’ is the autodiff method used by most deep learning frameworks, due to its efficiency and accuracy. Let’s: Look at how reverse-mode autodiff works. Create a minimal autodiff framework in Python. The small autodiff framework will deal with scalars. However we will look at a method of vectorising it with NumPy. We will also look at how

Explore this link on the map →

related reading