Denoising Density Estimators
Learning probabilistic models that can estimate the density of a given set of samples, and generate samples from that density, is one of the fundamental challenges in unsupervised machine learning. We introduce a new generative model based on denoising density estimators (DDEs), which are scalar functions parameterized by neural networks, that are efficiently trained to represent kernel density estimators of the data. Leveraging DDEs, our main contribution is a novel technique to obtain generative models by minimizing the KL-divergence directly. We prove that our algorithm for obtaining generative models is guaranteed to converge to the correct solution. Our approach does not require specific network architecture as in normalizing flows, nor use ordinary differential equation solvers as in continuous normalizing flows. Experimental results demonstrate substantial improvement in density estimation and competitive performance in generative model training.
Learning Generative Models using Denoising Density Estimators Siavash A. Bigdeli Geng Lin CSEM, Neuchâtel University of Maryland Switzerland College Park, USA arXiv:2001.02728v2 [cs.LG] 9 Jun 2020 Tiziano Portenier…
saved by
related reading
- Deep Energy Estimator.pdfarxiv.org
- Regularized by Score Matching (LeCun)papers.nips.cc
- [2007.13640] Solving Linear Inverse Problems Using the Prior Implicit in a Denoiserarxiv.org
- Noise2Scoreproceedings.neurips.cc
- Unsupervised Learning with SURE.pdfarxiv.org
- score matching (hyvarinen)jmlr.org
- Extending Stein's unbiased risk estimator to train deep denoisers with correlated pairs of noisy imagesproceedings.neurips.cc
- Noiser2Noise.pdfarxiv.org
- What are Diffusion Models? | Lil'Loglilianweng.github.io
- Yang Songyang-song.net
- The Principles of Diffusion Modelsarxiv.org
- The Principles of Diffusion Modelsarxiv.org