Azlen Elza
18 followers · 8 following · 641 views
on the atlas · 6
- Machiavelli I – S.P.Q.F. (Begins Machiavelli Series) – Ex Urbe1 savers
- On Progress and Historical Change – Ex Urbe3 savers
- Yes you should understand backprop | by Andrej Karpathy | Medium14 savers
- Naturally Occurring Equivariance in Neural Networks4 savers
- Zoom In: An Introduction to Circuits23 savers
- Curius5 savers
highlights · 6
Sometimes you can forward the entire training set through a trained network and find that a large fraction (e.g. 40%) of your neurons were zero the entire time.
Yes you should understand backprop | by Andrej Karpathy | MediumHue→Hue Circuit: Let’s start with a circuit connecting two sets of hue-equivariant center-surround detectors. Each unit in the second layer is excited by the unit selecting for a similar hue in the previous layer.
Naturally Occurring Equivariance in Neural NetworksRotational Equivariance: One example of equivariance is rotated versions of the same feature. These are especially common in early vision, for example curve detectors, high-low frequency detectors, and line detectors.
Naturally Occurring Equivariance in Neural NetworksUniversality (or “convergent learning”) of features has been suggested before. Prior work has shown that different neural networks can develop highly correlated neurons
Zoom In: An Introduction to Circuitsconsisting a set of tightly linked features and the weights between them
Zoom In: An Introduction to CircuitsTo be clear, this neuron isn’t responding to some commonality of cars and cat faces. Feature visualization shows us that it’s looking for the eyes and whiskers of a cat, for furry legs, and for shiny fronts of cars — not some subtle shared feature.
Zoom In: An Introduction to Circuits