Introduction to Neural Compression | Desi R. Ivanova

Machine learning, and deep probabilistic modelling specifically, seems to be revolutionising the space of data compression. This short post describes 1) the basic components of the data compression pipeline; 2) the objective used to optimise model parameters and its equivalence to training a VAE; and 3) some of the challenges that need to be solved.

Introduction to Neural Compression Learned neural methods for lossy image compression Desi R. Ivanova Last updated on 18 Feb, 2022 7 min read ML While searching for a research internship in industry, I discovered the exciting and rapidly growing field of neural data compression and decided to summarise what I've learned in this blog post. Specifically, I'll give a brief introduction to neural-based approaches for lossy compression , mainly focusing on images, though the methods discussed can be applied to other data modalities too. The compression pipeline The typical compression pipeline cons

related reading

• Compression and Intelligencegreene.sh
• Difference between AutoEncoder (AE) and Variational AutoEncoder (VAE) | Towards Data Sciencetowardsdatascience.com
• Generative modelling in latent space – Sander Dielemansander.ai
• [2309.10668] Language Modeling Is Compressionarxiv.org
• What are Diffusion Models? | Lil'Loglilianweng.github.io
• Tutorial - What is a variational autoencoder? – Jaan Lı 李jaan.io
• Compression is predictionngrok.com
• [1906.02691] An Introduction to Variational Autoencodersarxiv.org
• Tips for Training Likelihood Modelsblog.evjang.com
• The Little Book of Deep Learningfleuret.org
• Understanding VQ-VAE (DALL-E Explained Pt. 1)mlberkeley.substack.com
• https://arxiv.org/pdf/2006.11239arxiv.org