Efficiently Scaling Transformer Inference
arxiv.org · 5,072 words · saved by 1 readers
N/A
E FFICIENTLY S CALING T RANSFORMER I NFERENCE Reiner Pope 1 Sholto Douglas 1 Aakanksha Chowdhery 1 Jacob Devlin 1 James Bradbury 1 Anselm Levskaya 1 Jonathan Heek 1 Kefan Xiao 1 Shivani Agrawal 1 Jeff Dean 1 A BSTRACT We study the problem of efficient generative inference for Transformer models, in one of its most challenging…
related reading
- Transformer Inference Arithmetic | kipply's blogkipp.ly
- All About Transformer Inferencejax-ml.github.io
- How To Scale Your Modeljax-ml.github.io
- Transformers Inference Optimization Toolset | AstraBlogastralord.github.io
- Fast Inference from Transformers via Speculative Decodingarxiv.org
- Full Stack Optimization of Transformer Inference: a Surveyarxiv.org
- [2207.00032] DeepSpeed Inference: Enabling Efficient Inference of Transformer Models at Unprecedented Scalearxiv.org
- LLM Inference Performance Engineering: Best Practices | Databricks Blogdatabricks.com
- Transformer Math 101 | EleutherAI Blogblog.eleuther.ai
- Context Parallelism for Scalable Million-Token Inferencearxiv.org
- All the Transformer Math You Need to Know | How To Scale Your Modeljax-ml.github.io
- Reducing Activation Recomputationarxiv.org