Normal Approximation to the Posterior Distribution | Bounded Rationality
bjlkeng.github.io · 3,323 words · saved by 1 readers
A brief note on using a normal distribution to approximate a Bayesian posterior distribution.
In this post, I'm going to write about how the ever versatile normal distribution can be used to approximate a Bayesian posterior distribution. Unlike some other normal approximations, this is not a direct application of the central limit theorem. The result has a straight forward proof using Laplace's Method whose main ideas I will attempt to present. I'll also simulate a simple scenario to see how it works in practice. Background ¶ We're going to first start by reviewing some simple terminology and definitions regarding Bayesian methods to make the discussion later a bit easier to follow. Th
related reading
- Approximating KL Divergencejoschu.net
- Bayesian Neural Networkscs.toronto.edu
- Conjugate prior - Wikipediaen.wikipedia.org
- Coordinate Ascent Mean-field Variational Inference (Univariate Gaussian Example)suzyahyah.github.io
- Learning to be Bayesian without Supervisionpapers.nips.cc
- Bayesian programming - Wikipediaen.wikipedia.org
- bayesian - Bayes regression: how is it done in comparison to standard regression? - Cross Validatedstats.stackexchange.com
- Variational Inference with Normalizing Flowsarxiv.org
- RescaledKL.tmdanilorezende.com
- Maximum Likelihood, Fisher Information, Cramer Rao Inequalityomkar-ranadive.github.io
- Bayesian hierarchical modeling - Wikipediaen.wikipedia.org
- Laplace's method - Wikipediaen.wikipedia.org