Progressive Point Matching

Assigning partial credit to improve reinforcement learning for long-horizon reasoning tasks.

Preston Fu September 2026 Today’s LLMs tackle extremely long-horizon tasks that may run continuously for hours or days. Tasks that take humans days or weeks may require language model trajectories containing millions, or eventually billions, of tokens. These capabilities have been enabled by large-scale reinforcement learning (RL). The standard approach is to sample full trajectories and to assign a sparse outcome reward to the full trajectory – a 0 or 1 based on whether the trajectory was successful. Empirically, this simple approach has demonstrated stable performance improvements at…

saved by

• Akira Yoshiyama

related reading

• PPO for LLMs: A Guide for Normal Peoplecameronrwolfe.substack.com
• [2602.11399] Can We Really Learn One Representation to Optimize All Rewards?arxiv.org
• [2603.21383] PivotRL: High Accuracy Agentic Post-Training at Low Compute Costarxiv.org
• [2602.19362] LLMs Can Learn to Reason Via Off-Policy RLarxiv.org
• GRPO++: Tricks for Making RL Actually Workcameronrwolfe.substack.com
• State of RL for reasoning LLMs | A. Weersaweers.de
• [1707.06347] Proximal Policy Optimization Algorithmsarxiv.org
• [2510.13786] The Art of Scaling Reinforcement Learning Compute for LLMsarxiv.org
• RLHF | John Lambertjohnwlambert.github.io
• A vision researcher’s guide to some RL stuff: PPO & GRPO - Yuge (Jimmy) Shiyugeten.github.io
• Flow Matching Policy Gradientsflowreinforce.github.io
• [2305.18290] Direct Preference Optimization: Your Language Model is Secretly a Reward Modelarxiv.org