Training LLMs to Predict World Events (Guest Post with Mantic) - Thinking Machines Lab
Mantic have been using Tinker since it launched. This guest post is a technical deep dive on what they have built so far. The top AI forecasting systems are approaching superforecaster-level accuracy on geopolitics and current affairs.It’s over (Polymarket, 2026) This is exciting because scalable, automated forecasting could significantly improve the quality of decision-making across the economy and in government. To date, the most successful recipe in forecasting tournaments has been to combine an off-the-shelf LLM (like Gemini 3 or GPT-5) with forecasting-specific context-gathering. These models, to our knowledge, have not been explicitly trained for forecasting. Can we improve the recipe by replacing them with models fine-tuned specifically for forecasting?
Mantic have been using Tinker since it launched. This guest post is a technical deep dive on what they have built so far. The top AI forecasting systems are approaching superforecaster-level accuracy on geopolitics and current affairs. It’s over (Polymarket, 2026) This is exciting because scalable, automated forecasting could significantly improve the quality of decision-making across the economy and in government. To date, the most successful recipe in forecasting tournaments has been to combine an off-the-shelf LLM (like Gemini 3 or GPT-5) with forecasting-specific context-gathering. These m
saved by
related reading
- 🔮 IT’S OVER - The Oracle by Polymarketnews.polymarket.com
- Pitfalls in Evaluating Language Model Forecastersarxiv.org
- Predicting Empirical AI Research Outcomes with Language Modelsarxiv.org
- As Rocks May Think | Eric Jangevjang.com
- The AI Superforecasters Are Here - by Scott Alexanderastralcodexten.com
- LLMs Are Closing the Gap on Human Superforecasterssubstack.com
- Tinkerthinkingmachines.ai
- AI in 2025: gestalt — LessWronglesswrong.com
- Exploreforecastbench.org
- Patterns for Building LLM-based Systems & Productseugeneyan.com
- Modifying LLM Beliefs with Synthetic Document Finetuningalignment.anthropic.com
- What 2026 looks like — AI Alignment Forumalignmentforum.org