✳flâneur — a map of the web's best reading

Diffusion Language Models are Super Data Learners

jinjieni.notion.site · 249 words · saved by 1 readers

Skip to content Diffusion Language Models are Super Data Learners Get Notion free Diffusion Language Models are Super Data Learners Jinjie Ni and the team jinjieni@nus.edu.sg Released on Aug 09 2025 GitHub: dlms-are-super-data-learners​ Recent research highlights the potential of diffusion language models (DLMs). Owing to the parallel decoding design, they can generate thousands of tokens per second, resulting in exceptionally low latency for real-world applications [17][18][19]. Moreover, several recent DLMs have demonstrated performance on par with autoregressive (AR) models [8][9]. But is speed their only advantage? After rigorous investigations over the past few months, we discovered a more striking trait: diffusion models are super data learners under fixed data budgets. That is, given the same number of unique pre-training tokens, diffusion models consistently outperform AR counterparts of equal size—by trading additional FLOPs for improved learning. This reflects a roughly >3x d

Notion’s local storage may be damaged. See (?) > Help & documentation > Reset Notion. Contact support if that doesn’t fix the issue. Your Firefox profile may be damaged. Visit https://firefox-storage-test.glitch.me/ to diagnose. Contact support if that doesn’t fix the issue. Your Chrome profile may be damaged. If you changed any chrome://flags then please reset them, then restart your browser. If issues persist, try making a new Chrome user. Contact support if that doesn’t fix the issue. Your Chrome profile may be damaged. For a more consistent experience download the Notion desktop app: https

Explore this link on the map →

related reading