flâneur

jotterbach.github.io/content/posts/autoencoders/2016-07-18-AutoEncoders/

jotterbach.github.io · 1,943 words · saved by 1 readers

Quite in line with the last blog posts about Principal Component Analysis (PCA) and t-distributed Stochastic Neighbor Embedding (t-SNE) I want to discuss another dimensionality reduction technique that originated in the Neural Network (NN) community, known as Autoencoders. The idea behind an autoencoder is conceputally quite simple and results in very powerful outcomes if applied correctly. Unfortunately, however, it suffers from the typical problem of NNs in that the outcomes are not readily interpretablefn-images and somewhat harder to train. But before we jump into the autoencoder let's do some preparatory work for motivation and understanding. The code and and a notebook are available on my personal github. To illustrate the idea let's look at a simple model where we have two dimension that follow the relationship with a noise term 𝜖 ∼ 𝑁 ( 0 , 0.1 ) ϵ∼N(0,0.1), a bias 𝛼 α and a slope 𝛽 β as show in the figure to the left. To truthfully represent the full data we would have t

Unsupervised Representation Learning a.k.a. Autoencoders Quite in line with the last blog posts about Principal Component Analysis (PCA) and t-distributed Stochastic Neighbor Embedding (t-SNE) I want to discuss another dimensionality reduction technique that originated in the Neural Network (NN) community, known as Autoencoders . The idea behind an autoencoder is conceputally quite simple and results in very powerful outcomes if applied correctly. Unfortunately, however, it suffers from the typical problem of NNs in that the outcomes are not readily interpretable fn-images and somewhat harder

related reading