OpenThoughts3 - A new SOTA Reasoning Data Recipe | Open Thoughts

Pushing the boundaries of open reasoning datasets through rigorous experimentation.

OpenThinker3-7B is the SOTA open-data reasoning model at its scale. Our model achieves 53% on AIME 2025, 51% on LiveCodeBench 06/24-01/25, and 54% on GPQA Diamond, representing improvements of 15.3, 17.2, and 20.5 percentage points compared to DeepSeek-R1-Distill-Qwen-7B. All of our datasets and models are available on Hugging Face . We trained OpenThinker3-7B using only supervised fine-tuning, without any reinforcement learning. The key to our model’s performance is our new dataset, OpenThoughts3-1.2M . This dataset comprises 1.2 million questions across math, code, and science domains, with

related reading

• Qwen3: Think Deeper, Act Faster | Qwenqwenlm.github.io
• Inkling: Our Open-Weights Model - Thinking Machines Labthinkingmachines.ai
• Learning to reason with LLMs | OpenAIopenai.com
• A Safe Path to Open Weights - Thinking Machines Labthinkingmachines.ai
• DeepSeek-V3.2: Pushing the Frontier of Open Large Language Modelsarxiv.org
• DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Modelsarxiv.org
• [2201.11903] Chain of Thought Prompting Elicits Reasoning in Large Language Modelsarxiv.org
• Olmo 3: Charting a path through the model flow to lead open-source AI | Ai2allenai.org
• GitHub - openlm-research/open_llama: OpenLLaMA, a permissively licensed open source reproduction of Meta AI’s LLaMA 7B trained on the RedPajama datasetgithub.com
• [2203.14465] STaR: Bootstrapping Reasoning With Reasoningarxiv.org
• GitHub - open-compass/opencompass: OpenCompass is an LLM evaluation platform, supporting a wide range of models (Llama3, Mistral, InternLM2,GPT-4,LLaMa2, Qwen,GLM, Claude, etc) over 100+ datasets.github.com
• [2604.13010] Lightning OPD: Efficient Post-Training for Large Reasoning Models with Offline On-Policy Distillationarxiv.org