CS231n Convolutional Neural Networks for Visual Recognition

cs231n.github.io · 4,682 words · saved by 1 readers

It is possible to introduce neural networks without appealing to brain analogies. In the section on linear classification we computed scores for different visual categories given the image using the formula s=Wx 𝑠 = 𝑊 𝑥 , where W 𝑊 was a matrix and x 𝑥 was an input column vector containing all pixel data of the image. In the case of CIFAR-10, x 𝑥 is a [3072x1] column vector, and W 𝑊 is a [10x3072] matrix, so that the output scores is a vector of 10 class scores. An example neural network would instead compute s= W 2 max(0, W 1 x) 𝑠 = 𝑊 2 max ( 0 , 𝑊 1 𝑥 ) . Here, W 1 𝑊 1 could be, for example, a [100x3072] matrix transforming the image into a 100-dimensional intermediate vector. The function max(0,−) 𝑚 𝑎 𝑥 ( 0 , − ) is a non-linearity that is applied elementwise. There are several choices we could make for the non-linearity (which we’ll study below), but this one is a common choice and simply thresholds all activations that are below zero to zero. Finally,

Table of Contents: Quick intro without brain analogies Modeling one neuron Biological motivation and connections Single neuron as a linear classifier Commonly used activation functions Neural Network architectures Layer-wise organization Example feed-forward computation Representational power Setting number of layers and their sizes Summary Additional references Quick intro It is possible to introduce neural networks without appealing to brain analogies. In the section on linear classification we computed scores for different visual categories given the image using the formula \( s = W x \), w

CS231n Convolutional Neural Networks for Visual Recognition

related reading