Sparsity Blues

As measurements become more complex they often convolve meaningful phenomena with more extraneous phenomena. In order to limit the impact of these irrelevant phenomena on our inferences, and isolate the relevant phenomena, we need models that encourage sparse inferences. In this case study I review the basics of Bayesian inferential sparsity and some of the various strategies for building prior models that incorporate sparsity assumptions.

Sparsity Blues Sparsity Blues Michael Betancourt May 2021 As measurements become more complex they often convolve meaningful phenomena with more extraneous phenomena. In order to limit the impact of these irrelevant phenomena on our inferences, and isolate the relevant phenomena, we need models that encourage sparse inferences. In this case study I review the basics of Bayesian inferential sparsity and some of the various strategies for building prior models that incorporate sparsity assumptions. 1 Fading into Irrelevance Fundamentally the concept of sparsity requires some notion of a populati

related reading

• Writing - betanalpha.github.iobetanalpha.github.io
• Bayesian programming - Wikipediaen.wikipedia.org
• Bayesian Neural Networkscs.toronto.edu
• Weight-Sparse Circuits May Be Interpretable Yet Unfaithful — LessWronglesswrong.com
• Overview of Sparse Modeling | Chan`s Jupytergoodboychan.github.io
• Hierarchical Modelingbetanalpha.github.io
• Sparse Attention Post-Training for Mechanistic Interpretabilityarxiv.org
• Gregory Gundersengregorygundersen.com
• Bryon Aragam // University of Chicagobryonaragam.com
• L1 regularization: sparsity through singularitiesejenner.com
• Researchbactra.org
• Bayesian network - Wikipediaen.wikipedia.org