What Makes Low-Bit Quantization-Aware Training Work for Reasoning LLMs? A Systematic Study
This is experimental HTML to improve accessibility. We invite you to report rendering errors. Use Alt+Y to toggle on accessible reporting links and Alt+Shift+Y to toggle off. Learn more about this project and help improve conversions. Reasoning models excel at complex tasks such as coding and mathematics, yet their inference is often slow and token-inefficient. To improve the inference efficiency, post-training quantization (PTQ) usually comes with the cost of large accuracy drops, especially for reasoning tasks under low-bit settings. In this study, we present a systematic empirical study of quantization-aware training (QAT) for reasoning models. Our key findings include: (1) Knowledge distillation is a robust objective for reasoning models trained via either supervised fine-tuning or reinforcement learning; (2) PTQ provides a strong initialization for QAT, improving accuracy while reducing training cost; (3) Reinforcement learning remains feasible for quantized models given a viable
What Makes Low-Bit Quantization-Aware Training Work for Reasoning LLMs? A Systematic Study Keyu Lv 1∗ , Manyi Zhang 2 , Xiaobo Xia 3 , Jingchen Ni 1 , Shannan Yan 1 , Xianzhi Yu 2 , Lu Hou 2 , Chun Yuan 1# , Haoli Bai 2# 1 Shenzhen International Graduate School, Tsinghua University 2 Huawei Technologies 3 National University of Singapore lvky24@mails.tsinghua.edu.cn yuanc@sz.tsinghua.edu.cn baihaoli@huawei.com Equal contribution. # Corresponding authors. Abstract Reasoning models excel at complex tasks such as coding and mathematics, yet their inference is often slow and token-inefficient. To
saved by
related reading
- You Only Need Minimal RLVR Training: Extrapolating LLMs via Rank-1 Trajectoriesarxiv.org
- [2510.11696] QeRL: Beyond Efficiency -- Quantization-enhanced Reinforcement Learning for LLMsarxiv.org
- [2510.13786] The Art of Scaling Reinforcement Learning Compute for LLMsarxiv.org
- Efficient LLM inferencefinbarrtimbers.substack.com
- 2305.14314arxiv.org
- Quantization from the ground upngrok.com
- Training Large Language Models to Reason in a Continuous Latent Spacearxiv.org
- Why Does Reinforcement Learning Generalize? A Feature-Level Mechanistic Study of Post-Training in Large Language Modelsarxiv.org
- [2506.19733] Breaking Barriers: Do Reinforcement Post Training Gains Transfer To Unseen Domains?arxiv.org
- [2504.13837] Does Reinforcement Learning Really Incentivize Reasoning Capacity in LLMs Beyond the Base Model?arxiv.org
- [2604.13010] Lightning OPD: Efficient Post-Training for Large Reasoning Models with Offline On-Policy Distillationarxiv.org
- Reasoning in General Domains without Verifiersarxiv.org