Dealing with Overconfidence in Neural Networks: Bayesian Approach – Jonathan Ramkissoon

I trained a classifier on images of animals and gave it an image of myself, it's 98% confident I'm a dog. This is an exploration of a possible Bayesian fix. Code available too

I trained a multi-class classifier on images of cats, dogs and wild animals and passed an image of myself, it’s 98% confident I’m a dog. The problem isn’t that I passed an inappropriate image, because models in the real world are passed all sorts of garbage. It’s that the model is overconfident about an image far away from the training data. Instead we expect a more uniform distribution over the classes. The overconfidence makes it difficult to post-process model output (setting a threshold on predictions, etc.), which means it needs to be dealt with by the architecture. In this post I explore

related reading

• On Getting Confidence Estimates from Neural Networks | Bharath's notesbharathpbhat.github.io
• Bayesian Neural Networkscs.toronto.edu
• [1706.04599] On Calibration of Modern Neural Networksarxiv.org
• A Recipe for Training Neural Networkskarpathy.github.io
• Cycles of Thought: Measuring LLM Confidence through Stable Explanationsarxiv.org
• arxiv.org/pdf/1805.08522arxiv.org
• A Recipe for Training Neural Networkskarpathy.github.io
• How confessions can keep language models honest | OpenAIopenai.com
• Teaching LLMs to reason like Bayesiansresearch.google
• Maybe I was too harsh on deep learning theory (three days ago) — LessWronglesswrong.com
• A Comprehensive Introduction to Bayesian Deep Learning - Joris Baanjorisbaan.nl
• Towards Data Sciencetowardsdatascience.com