Full Stack Optimization of Transformer Inference: a Survey
arxiv.org · 4,916 words · saved by 1 readers
N/A
Full Stack Optimization of Transformer Inference: a Survey Sehoon Kim∗ Coleman Hooper∗ Thanakul Wattanawong sehoonkim@berkeley.edu chooper@berkeley.edu j.wat@berkeley.edu UC Berkeley UC Berkeley UC Berkeley…
related reading
- transformer_attention.pdfarxiv.org
- A Mathematical Framework for Transformer Circuitstransformer-circuits.pub
- Transformer Inference Arithmetic | kipply's blogkipp.ly
- The Annotated Transformernlp.seas.harvard.edu
- Transformers Inference Optimization Toolset | AstraBlogastralord.github.io
- Efficiently Scaling Transformer Inferencearxiv.org
- In-context Learning and Induction Headstransformer-circuits.pub
- All About Transformer Inferencejax-ml.github.io
- Fast Inference from Transformers via Speculative Decodingarxiv.org
- Transformer Math 101 | EleutherAI Blogblog.eleuther.ai
- 1706.03762arxiv.org
- [2207.00032] DeepSpeed Inference: Enabling Efficient Inference of Transformer Models at Unprecedented Scalearxiv.org