Byte Latent Transformer: Patches Scale Better Than Tokens | Research - AI at Meta

We introduce the Byte Latent Transformer (BLT), a new byte-level LLM architecture that, for the first time, matches tokenization-based LLM performance at...

Byte Latent Transformer: Patches Scale Better Than Tokens | Research - AI at Meta Products AI Research Resources About Meta Model API Try Meta AI NLP Byte Latent Transformer: Patches Scale Better Than Tokens December 12, 2024 Abstract We introduce the Byte Latent Transformer (BLT), a new byte-level LLM architecture that, for the first time, matches tokenization-based LLM performance at scale with significant improvements in inference efficiency and robustness. BLT encodes bytes into dynamically sized patches, which serve as the primary units of computation. Patches are segmented dynamically ba

related reading

• Byte Latent Transformer: Patches Scale Better Than Tokensarxiv.org
• 2025.acl-long.453.pdfaclanthology.org
• The Bitter Lesson is coming for Tokenization – ⛰️ lucalplucalp.dev
• The End of Tokenizationronaldyu.substack.com
• LLM Inference Performance Engineering: Best Practices | Databricks Blogdatabricks.com
• Transformer Explainer: LLM Transformer Model Visually Explainedpoloclub.github.io
• Meta AI Unleashes Megabyte, a Revolutionary Scalable Model Architecture - Artisanaartisana.ai
• [2510.13786] The Art of Scaling Reinforcement Learning Compute for LLMsarxiv.org
• The Era of 1-bit LLMs: All Large Language Models are in 1.58 Bitsarxiv.org
• Tensor-Transformer Variants are Surprisingly Performant — LessWronglesswrong.com
• Pathways Language Model (PaLM): Scaling to 540 Billion Parameters for Breakthrouai.googleblog.com
• [2512.23675] End-to-End Test-Time Training for Long Contextarxiv.org