✳flâneur — a map of the web's best reading

[2602.06019] Multi-Token Prediction via Self-Distillation

arxiv.org · 616 words · saved by 1 readers

Abstract:Existing techniques for accelerating language model inference, such as speculative decoding, require training auxiliary speculator models and building and deploying complex inference pipelines. We consider a new approach for converting a pretrained autoregressive language model from a slow single next token prediction model into a fast standalone multi-token prediction model using a simple online distillation objective. The final model retains the exact same implementation as the pretrained initial checkpoint and is deployable without the addition of any auxiliary verifier or other specialized inference code. On GSM8K, our method produces models that can decode more than $3\times$ faster on average at $<5\%$ drop in accuracy relative to single token decoding performance.

[2602.06019] Multi-Token Prediction via Self-Distillation Skip to main content arXiv is now an independent nonprofit! Learn more &times; Search arXiv Press Enter to search &middot; Advanced search --> Computer Science > Computation and Language arXiv:2602.06019 (cs) [Submitted on 5 Feb 2026 ( v1 ), last revised 23 Apr 2026 (this version, v2)] Title: Multi-Token Prediction via Self-Distillation Authors: John Kirchenbauer , Abhimanyu Hans , Brian Bartoldson , Micah Goldblum , Ashwinee Panda , Tom Goldstein View a PDF of the paper titled Multi-Token Prediction via Self-Distillation, by John Kirch

Explore this link on the map →

related reading