Progressive Point Matching

Assigning partial credit to improve reinforcement learning for long-horizon reasoning tasks.

Preston Fu September 2026 Today’s LLMs tackle extremely long-horizon tasks that may run continuously for hours or days. Tasks that take humans days or weeks may require language model trajectories containing millions, or eventually billions, of tokens. These capabilities have been enabled by large-scale reinforcement learning (RL). The standard approach is to sample full trajectories and to assign a sparse outcome reward to the full trajectory – a 0 or 1 based on whether the trajectory was successful. Empirically, this simple approach has demonstrated stable performance improvements at…

saved by

• Akira Yoshiyama

related reading

• PPO for LLMs: A Guide for Normal Peoplecameronrwolfe.substack.com
• GRPO++: Tricks for Making RL Actually Workcameronrwolfe.substack.com
• [2603.21383] PivotRL: High Accuracy Agentic Post-Training at Low Compute Costarxiv.org
• State of RL for reasoning LLMs | A. Weersaweers.de
• [2602.19362] LLMs Can Learn to Reason Via Off-Policy RLarxiv.org
• RLHF | John Lambertjohnwlambert.github.io
• Is Frontier Asynchronous RL Solved? — Luke J. Huangluk-huang.github.io
• DeepSeek-R1arxiv.org
• The State of Reinforcement Learning for LLM Reasoningmagazine.sebastianraschka.com
• Google Scholarscholar.google.com
• A vision researcher’s guide to some RL stuff: PPO & GRPO - Yuge (Jimmy) Shiyugeten.github.io
• [2605.03327] DGPO: Distribution Guided Policy Optimization for Fine Grained Credit Assignmentarxiv.org