Building Hamiltonian Monte Carlo (HMC) from Scratch: A Step-by-Step Guide | by Fortunato Nucera | Medium

In a previous post on this blog, I explained how to use the quadratic approximation to sample from the approximate posterior distribution of a Bayesian Linear Model. The quadratic approximation is an excellent technique which is often overlooked. In some scenarios though, especially in the case of hierarchical models, the quadratic approximation may not work. Markov Chain Monte Carlo (MCMC) does not make any assumptions on the posterior distribution, which can then become as flexible as we need it to be. We pay a high price for that though: harder implementation and slower sampling. The “harder implementation” is — as always — relative to the observer: the world is full of smart people! The “slower sampling” is, instead, something to consider more carefully. MCMC is a family of techniques which comprises very different samplers. I wrote about how to sample using Metropolis-Hastings (MH) in this post. At the same time, the convergence rate of MH may be rather slow, thus demanding a more

In a previous post on this blog, I explained how to use the quadratic approximation to sample from the approximate posterior distribution of a Bayesian Linear Model. The quadratic approximation is an excellent technique which is often overlooked. In some scenarios though, especially in the case of hierarchical models, the quadratic approximation may not work. Markov Chain Monte Carlo (MCMC) does not make any assumptions on the posterior distribution, which can then become as flexible as we need it to be. We pay a high price for that though: harder implementation and slower sampling. The “harde