Training LLMs to Predict World Events (Guest Post with Mantic) - Thinking Machines Lab
Mantic have been using Tinker since it launched. This guest post is a technical deep dive on what they have built so far. The top AI forecasting systems are approaching superforecaster-level accuracy on geopolitics and current affairs.It’s over (Polymarket, 2026) This is exciting because scalable, automated forecasting could significantly improve the quality of decision-making across the economy and in government. To date, the most successful recipe in forecasting tournaments has been to combine an off-the-shelf LLM (like Gemini 3 or GPT-5) with forecasting-specific context-gathering. These models, to our knowledge, have not been explicitly trained for forecasting. Can we improve the recipe by replacing them with models fine-tuned specifically for forecasting?
Mantic have been using Tinker since it launched. This guest post is a technical deep dive on what they have built so far. The top AI forecasting systems are approaching superforecaster-level accuracy on geopolitics and current affairs. It’s over (Polymarket, 2026) This is exciting because scalable, automated forecasting could significantly improve the quality of decision-making across the economy and in government. To date, the most successful recipe in forecasting tournaments has been to combine an off-the-shelf LLM (like Gemini 3 or GPT-5) with forecasting-specific context-gathering. These m
saved by
related reading
- 🔮 IT’S OVER - The Oracle by Polymarketnews.polymarket.com
- On-Policy Distillation - Thinking Machines Labthinkingmachines.ai
- [2510.13786] The Art of Scaling Reinforcement Learning Compute for LLMsarxiv.org
- Learning to Replicate Expert Judgment in Financial Tasks - Thinking Machines Labthinkingmachines.ai
- Simulators — LessWronglesswrong.com
- Modifying LLM Beliefs with Synthetic Document Finetuningalignment.anthropic.com
- LLMs Are Closing the Gap on Human Superforecasterssubstack.com
- The AI Superforecasters Are Here - by Scott Alexanderastralcodexten.com
- Anatomy of a Modern Finetuning APIbenanderson.work
- Tinkerthinkingmachines.ai
- Pitfalls in Evaluating Language Model Forecastersarxiv.org
- What will GPT-2030 look like? — AI Alignment Forumalignmentforum.org