The State of Reinforcement Learning for LLM Reasoning

Understanding GRPO and New Insights from Reasoning Model Papers

The State of Reinforcement Learning for LLM Reasoning Understanding GRPO and New Insights from Reasoning Model Papers Sebastian Raschka, PhD Apr 19, 2025 518 35 40 Share A lot has happened this month, especially with the releases of new flagship models like GPT-4.5 and Llama 4. But you might have noticed that reactions to these releases were relatively muted. Why? One reason could be that GPT-4.5 and Llama 4 remain conventional models, which means they were trained without explicit reinforcement learning for reasoning. Meanwhile, competitors such as xAI and Anthropic have added more reasoning

saved by

• Jeff Brown

related reading

• A Guide to Reinforcement Learning Post-Training for LLMs: PPO, DPO, GRPO, and Beyondhuggingface.co
• [2603.21972] Demystifying Reinforcement Learning for Long-Horizon Tool-Using Agents: A Comprehensive Recipearxiv.org
• The State of Reinforcement Learning for LLM Reasoningsebastianraschka.com
• State of RL for reasoning LLMs | A. Weersaweers.de
• A vision researcher’s guide to some RL stuff: PPO & GRPO - Yuge (Jimmy) Shiyugeten.github.io
• DeepSeek-R1arxiv.org
• GRPO++: Tricks for Making RL Actually Workcameronrwolfe.substack.com
• Illustrating Reinforcement Learning from Human Feedback (RLHF)huggingface.co
• Understanding Reasoning LLMs - by Sebastian Raschka, PhDsebastianraschka.com
• [2510.13786] The Art of Scaling Reinforcement Learning Compute for LLMsarxiv.org
• RLHF: Reinforcement Learning from Human Feedbackhuyenchip.com
• The State of LLM Reasoning Model Inferencesebastianraschka.com