✳flâneur — a map of the web's best reading

From Autoencoder to Beta-VAE | Lil'Log

lilianweng.github.io · 4,362 words · saved by 1 readers

[Updated on 2019-07-18: add a section on VQ-VAE & VQ-VAE-2.] [Updated on 2019-07-26: add a section on TD-VAE.] Autocoder is invented to reconstruct high-dimensional data using a neural network model with a narrow bottleneck layer in the middle (oops, this is probably not true for Variational Autoencoder, and we will investigate it in details in later sections). A nice byproduct is dimension reduction: the bottleneck layer captures a compressed latent encoding. Such a low-dimensional representation can be used as en embedding vector in various applications (i.e. search), help data compression, or reveal the underlying data generative factors. Autoencoder is a neural network designed to learn an identity function in an unsupervised way to reconstruct the original input while compressing the data in the process so as to discover a more efficient and compressed representation. The idea was originated in the 1980s, and later promoted by the seminal paper by Hinton & Salakhutdinov, 2006. It

Table of Contents Notation Autoencoder Denoising Autoencoder Sparse Autoencoder Contractive Autoencoder VAE: Variational Autoencoder Loss Function: ELBO Reparameterization Trick Beta-VAE VQ-VAE and VQ-VAE-2 TD-VAE References [Updated on 2019-07-18: add a section on VQ-VAE & VQ-VAE-2 .] [Updated on 2019-07-26: add a section on TD-VAE .] Autocoder is invented to reconstruct high-dimensional data using a neural network model with a narrow bottleneck layer in the middle (oops, this is probably not true for Variational Autoencoder , and we will investigate it in details in later sections). A nice b

Explore this link on the map →

related reading