High-Dimensional Gaussian Sampling: A Review and a Unifying Approach Based on a Stochastic Proximal Point Algorithm
Efficient sampling from a high-dimensional Gaussian distribution is an old but high-stake issue. Vanilla Cholesky samplers imply a computational cost and memory requirements which can rapidly become prohibitive in high dimension. To tackle these issues, multiple methods have been proposed from different communities ranging from iterative numerical linear algebra to Markov chain Monte Carlo (MCMC) approaches. Surprisingly, no complete review and comparison of these methods have been conducted. This paper aims at reviewing all these approaches by pointing out their differences, close relations, benefits and limitations. In addition to this state of the art, this paper proposes a unifying Gaussian simulation framework by deriving a stochastic counterpart of the celebrated proximal point algorithm in optimization. This framework offers a novel and unifying revisit of most of the existing MCMC approaches while extending them. Guidelines to choose the appropriate Gaussian simulation method for a given sampling problem in high dimension are proposed and illustrated with numerical examples.
HIGH-DIMENSIONAL GAUSSIAN SAMPLING: A REVIEW AND A UNIFYING APPROACH BASED ON A STOCHASTIC PROXIMAL POINT ALGORITHM MAXIME VONO∗ , NICOLAS DOBIGEON† , AND PIERRE CHAINAIS‡ Abstract. Efficient sampling from a high-dimensional Gaussian distribution is an old but high- stake issue. Vanilla Cholesky samplers imply a computational cost and memory…
saved by
related reading
- Scalable Gradients for Stochastic Differential Equationsarxiv.org
- Stochastic Modified Equations and Dynamics of Stochastic Gradient Algorithms I: Mathematical Foundationsjmlr.org
- An Introduction to Malliavin Calculusuni-ulm.de
- Partial Recovery and Weak Consistency in the Non-Uniform Hypergraph Stochastic Block Modelarxiv.org
- Analysis of Circulant Embedding Methods for Sampling Stationary Random Fieldsarxiv.org
- A Primer on Stochastic Partial Differential Equationsmath.utah.edu
- An optimization perspective on log-concave sampling and beyond | Sinho Chewichewisinho.github.io
- Step-by-Step Diffusion: An Elementary Tutorialarxiv.org
- [2201.05002] Boost your favorite Markov Chain Monte Carlo sampler using Kac's theorem: the Kick-Kac teleportation algorithmarxiv.org
- [2506.17366] Gaussian Processes and Reproducing Kernel Hilbert Spaces: Connections and Equivalencesarxiv.org
- Sampling: Two Basic Algorithmsgregorygundersen.com
- A Review of Modern Stochastic Modeling: SDE/SPDE Numerics, Data-Driven Identification, and Generative Methods with Applications in Biology and Epidemiologyarxiv.org