CS231n Convolutional Neural Networks for Visual Recognition

It is possible to introduce neural networks without appealing to brain analogies. In the section on linear classification we computed scores for different visual categories given the image using the formula s=Wx 𝑠 = 𝑊 𝑥 , where W 𝑊 was a matrix and x 𝑥 was an input column vector containing all pixel data of the image. In the case of CIFAR-10, x 𝑥 is a [3072x1] column vector, and W 𝑊 is a [10x3072] matrix, so that the output scores is a vector of 10 class scores. An example neural network would instead compute s= W 2 max(0, W 1 x) 𝑠 = 𝑊 2 max ( 0 , 𝑊 1 𝑥 ) . Here, W 1 𝑊 1 could be, for example, a [100x3072] matrix transforming the image into a 100-dimensional intermediate vector. The function max(0,−) 𝑚 𝑎 𝑥 ( 0 , − ) is a non-linearity that is applied elementwise. There are several choices we could make for the non-linearity (which we’ll study below), but this one is a common choice and simply thresholds all activations that are below zero to zero. Finally,

Table of Contents: Quick intro without brain analogies Modeling one neuron Biological motivation and connections Single neuron as a linear classifier Commonly used activation functions Neural Network architectures Layer-wise organization Example feed-forward computation Representational power Setting number of layers and their sizes Summary Additional references Quick intro It is possible to introduce neural networks without appealing to brain analogies. In the section on linear classification we computed scores for different visual categories given the image using the formula \( s = W x \), w

related reading

• Neural networks and deep learningneuralnetworksanddeeplearning.com
• Convolutional Neural Networks, Explained | Towards Data Sciencetowardsdatascience.com
• CS231n Deep Learning for Computer Visioncs231n.github.io
• Zoom In: An Introduction to Circuitsdistill.pub
• Toy Models of Superpositiontransformer-circuits.pub
• Neural Networks, Manifolds, and Topology -- colah's blogcolah.github.io
• CS231n Deep Learning for Computer Visioncs231n.github.io
• A Recipe for Training Neural Networkskarpathy.github.io
• A Beginner's Guide To Understanding Convolutional Neural Networks – Adit Deshpande – Engineering at Forward | UCLA CS '19adeshpande3.github.io
• Neural Networks, Types, and Functional Programming -- colah's blogcolah.github.io
• Feature Visualizationdistill.pub
• Neural networks and deep learningneuralnetworksanddeeplearning.com