Sparsity Blues

As measurements become more complex they often convolve meaningful phenomena with more extraneous phenomena. In order to limit the impact of these irrelevant phenomena on our inferences, and isolate the relevant phenomena, we need models that encourage sparse inferences. In this case study I review the basics of Bayesian inferential sparsity and some of the various strategies for building prior models that incorporate sparsity assumptions.

Sparsity Blues Sparsity Blues Michael Betancourt May 2021 As measurements become more complex they often convolve meaningful phenomena with more extraneous phenomena. In order to limit the impact of these irrelevant phenomena on our inferences, and isolate the relevant phenomena, we need models that encourage sparse inferences. In this case study I review the basics of Bayesian inferential sparsity and some of the various strategies for building prior models that incorporate sparsity assumptions. 1 Fading into Irrelevance Fundamentally the concept of sparsity requires some notion of a populati

related reading

• Hierarchical Modelingbetanalpha.github.io
• Bayesian Neural Networkscs.toronto.edu
• Learning to be Bayesian without Supervisionpapers.nips.cc
• Musings on typicality – Sander Dielemansander.ai
• [2304.11751] Score-Based Diffusion Models as Principled Priors for Inverse Imagingarxiv.org
• Sparse Attention Post-Training for Mechanistic Interpretabilityarxiv.org
• Sparse Autoencoders Find Highly Interpretable Features in Language Modelsarxiv.org
• 2012.00174arxiv.org
• Bayesian programming - Wikipediaen.wikipedia.org
• Writing - betanalpha.github.iobetanalpha.github.io
• catoni_draft2.pdfocatoni.perso.math.cnrs.fr
• Flow-based likelihoods.pdfarxiv.org