✳flâneur — a map of the web's best reading

Sleep is for backpropagation

dimitarsimeonov.com · 884 words · saved by 1 readers

Diving deeper and deeper into machine learning and neural networks I can’t help but notice how some of the techniques there are metaphors for what happens in our brains. After all, our brains a just a lot more sophisticated and larger versions of the deep computer networks that do computer vision, or predict clicks on ads. When I first grokked backpropagation, I was in awe about how elegant it is. An iterative approach to computing the partial derivative of multi-dimensional, complex probability density function. Beauty, in the mathematical sense of the word. Then, as I learned about the more practical aspects of it, I learned about the less intuitive concept of “batch_size”. Every round of backpropagation makes the neural network slightly better at its task, by updating the individual neurons based on the errors they make. The batch size comes from the algorighm known as SGD which will only look at small amount of examples before updating. It makes sense to look at the data one exampl

Sleep is for backpropagation --> --> Sleep is for backpropagation 22 December 2017 Diving deeper and deeper into machine learning and neural networks I can’t help but notice how some of the techniques there are metaphors for what happens in our brains. After all, our brains a just a lot more sophisticated and larger versions of the deep computer networks that do computer vision, or predict clicks on ads. When I first grokked backpropagation, I was in awe about how elegant it is. An iterative approach to computing the partial derivative of multi-dimensional, complex probability density function

Explore this link on the map →

related reading