[2203.14465] STaR: Bootstrapping Reasoning With Reasoning
Abstract:Generating step-by-step "chain-of-thought" rationales improves language model performance on complex reasoning tasks like mathematics or commonsense question-answering. However, inducing language model rationale generation currently requires either constructing massive rationale datasets or sacrificing accuracy by using only few-shot inference. We propose a technique to iteratively leverage a small number of rationale examples and a large dataset without rationales, to bootstrap the ability to perform successively more complex reasoning. This technique, the "Self-Taught Reasoner" (STaR), relies on a simple loop: generate rationales to answer many questions, prompted with a few rationale examples; if the generated answers are wrong, try again to generate a rationale given the correct answer; fine-tune on all the rationales that ultimately yielded correct answers; repeat. We show that STaR significantly improves performance on multiple datasets compared to a model fine-tuned to directly predict final answers, and performs comparably to fine-tuning a 30$\times$ larger state-of-the-art language model on CommensenseQA. Thus, STaR lets a model improve itself by learning from its own generated reasoning.
STaR: Self-Taught Reasoner Bootstrapping Reasoning With Reasoning Eric Zelikman∗1 , Yuhuai Wu∗12 , Jesse Mu1 , Noah D. Goodman1 1 Department of Computer Science, Stanford University 2…
saved by
related reading
- [2201.11903] Chain of Thought Prompting Elicits Reasoning in Large Language Modelsarxiv.org
- [2603.07267] How to Steal Reasoning Without Reasoning Tracesarxiv.org
- [2005.11401] Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasksarxiv.org
- Training Large Language Models to Reason in a Continuous Latent Spacearxiv.org
- Language Models can Solve Computer Tasksarxiv.org
- Base Models Know How to Reason, Thinking Models Learn Whenarxiv.org
- Externalized reasoning oversight: a research direction for language model alignment — AI Alignment Forumalignmentforum.org
- [2511.08579] Training Language Models to Explain Their Own Computationsarxiv.org
- Reasoning in General Domains without Verifiersarxiv.org
- [2601.18734] Self-Distilled Reasoner: On-Policy Self-Distillation for Large Language Modelsarxiv.org
- Self-Distilled Reasoner: On-Policy Self-Distillation for Large Language Modelsarxiv.org
- Thinking to recall: How reasoning unlocks parametric knowledge in LLMsresearch.google