Detecting when LLMs are Uncertain • Thariq Shihipar

A deep dive into a new reasoning technique called Entropix.

Detecting when LLMs are Uncertain Thariq Shihipar - 11 October 2024 · 7 min read This post tries to explain the new reasoning techniques developed by XJDR in a new project called Entropix . Entropix attempts to improve reasoning in models through being smarter at sampling during moments of uncertainty. A big caveat, there have been no large scale evals yet for Entropix, so it’s not clear how much this helps in practice. But it does seem to introduce some promising techniques and mental models for reasoning. Uncertainity at a glance Sampling is the process of choosing which token from the distr

related reading

• Cycles of Thought: Measuring LLM Confidence through Stable Explanationsarxiv.org
• Modifying LLM Beliefs with Synthetic Document Finetuningalignment.anthropic.com
• Thought Branches: Interpreting LLM Reasoning Requires Resamplingarxiv.org
• State of RL for reasoning LLMs | A. Weersaweers.de
• The Unintelligibility is Ours: Notes on Chain of Thought1a3orn.com
• How LLMs Work, Explained Without Math - miguelgrinberg.comblog.miguelgrinberg.com
• [2506.01939] Beyond the 80/20 Rule: High-Entropy Minority Tokens Drive Effective Reinforcement Learning for LLM Reasoningarxiv.org
• LLM Samplers Explained · GitHubgist.github.com
• Language Modelinglena-voita.github.io
• Taking LLMs Seriously (As Language Models) — LessWronglesswrong.com
• Interruption is All You Need: Reducing LLM Hallucination through Parallel Reasoning Diversity | David Baidavidbai.dev
• Reasoning as Trajectoriesslhleosun.github.io