On Getting Confidence Estimates from Neural Networks | Bharath's notes

I've been reading on and off about this topic for a while now, and it comes up very often in applied ML. How confident is the neural network about a particular prediction it makes? Can we switch to a different end user experience when the network is not confident? In a typical scenario, a model is trained on a limited dataset and gets deployed in a product, where unless the user experience is strictly guarded, the model will get hit with inputs that were not anticipated. This is the problem of Out-Of-Distribution (OOD) detection - and if a model can either say that it does not "know" what to do in such cases, or equivalently, make a low confidence prediction - then that is useful for avoiding catastrophic failures. There is a lot of research that focuses on this topic, and what follows is a summary of some of the papers I've read. So how do we measure the confidence of a neural network? The majority of research focuses on Classification models, so we'll talk about them first, with a se

I've been reading on and off about this topic for a while now, and it comes up very often in applied ML. How confident is the neural network about a particular prediction it makes? Can we switch to a different end user experience when the network is not confident? In a typical scenario, a model is trained on a limited dataset and gets deployed in a product, where unless the user experience is strictly guarded, the model will get hit with inputs that were not anticipated. This is the problem of Out-Of-Distribution (OOD) detection - and if a model can either say that it does not "know" what to d

saved by

• 4764

related reading

• [1706.04599] On Calibration of Modern Neural Networksarxiv.org
• Dealing with Overconfidence in Neural Networks: Bayesian Approach – Jonathan Ramkissoonjramkiss.github.io
• On Calibration of Modern Neural Networksproceedings.mlr.press
• Bayesian Neural Networkscs.toronto.edu
• A Recipe for Training Neural Networkskarpathy.github.io
• Cycles of Thought: Measuring LLM Confidence through Stable Explanationsarxiv.org
• A Recipe for Training Neural Networkskarpathy.github.io
• Feature Visualizationdistill.pub
• How confessions can keep language models honest | OpenAIopenai.com
• Confidence all the way upmindingourway.com
• [2003.07892] Calibration of Pre-trained Transformersarxiv.org
• arxiv.org/pdf/1805.08522arxiv.org