Bayesian Neural Networks

Bayesian inference allows us to learn a probability distribution over possible neural networks. We can approximately solve inference with a simple modification to standard neural network tools. The resulting algorithm mitigates overfitting, enables learning from small datasets, and tells us how uncertain our predictions are. You may have heard deep neural networks described as powerful function approximators. Their power is due to the extreme flexibility of having many model parameters (the weights and biases) whose values can be learned from data via gradient-based optimization. Because they are good at approximating functions (input-output relationships) when lots of data are available, neural networks are well-suited to artificial intelligence tasks like speech recognition and image classification. But the extreme flexibility of neural networks has a downside: they are particularly vulnerable to overfitting. Overfitting happens when the learning algorithm does such a good job of tun

Bayesian Neural Networks Bayesian Neural Networks Bayesian inference allows us to learn a probability distribution over possible neural networks. We can approximately solve inference with a simple modification to standard neural network tools. The resulting algorithm mitigates overfitting, enables learning from small datasets, and tells us how uncertain our predictions are. Train What's wrong with neural networks? You may have heard deep neural networks described as powerful function approximators . Their power is due to the extreme flexibility of having many model parameters (the weights and

saved by

• Rebecca Lin
• Claire Wang
• Ethan Kuntz
• Yudhister Joel Kumar
• mesityl
• Didier Lampard
• Akira Yoshiyama
• Ishan Mukherjee

related reading

• A Recipe for Training Neural Networkskarpathy.github.io
• nn-notes.pdfboris-hanin.github.io
• arxiv.org/pdf/1805.08522arxiv.org
• A Comprehensive Introduction to Bayesian Deep Learning - Joris Baanjorisbaan.nl
• Coordinate Ascent Mean-field Variational Inference (Univariate Gaussian Example)suzyahyah.github.io
• Understanding Deep Learningudlbook.github.io
• [1911.08731] Distributionally Robust Neural Networks for Group Shifts: On the Importance of Regularization for Worst-Case Generalizationarxiv.org
• QFT and neural nets: the basic idea — LessWronglesswrong.com
• Deep Ensembles as Approximate Bayesian Inference | Deep Ensembles as Approximate Bayesian Inferencecims.nyu.edu
• Neural networks and deep learningneuralnetworksanddeeplearning.com
• [2403.04807] Mathematics of Neural Networks (Lecture Notes Graduate Course)arxiv.org
• David Duvenaudcs.toronto.edu