Brainformers: Trading Simplicity for Efficiency
arxiv.org · 5,735 words · saved by 1 readers
N/A
Brainformers: Trading Simplicity for Efficiency Yanqi Zhou 1 Nan Du 1 Yanping Huang 1 Daiyi Peng 1 Chang Lan 1 Da Huang 1 Siamak Shakeri 1 David So 1 Andrew Dai 1 Yifeng Lu 1 Zhifeng Chen 1 Quoc Le 1 Claire Cui 1 James Laudon 1 Jeff Dean 1 Abstract Scaling…
related reading
- A Mathematical Framework for Transformer Circuitstransformer-circuits.pub
- transformer_attention.pdfarxiv.org
- GLaM: Efficient Scaling of Language Models with Mixture-of-Expertsarxiv.org
- Full Stack Optimization of Transformer Inference: a Surveyarxiv.org
- [2011.04006] Long Range Arena: A Benchmark for Efficient Transformersarxiv.org
- Efficiently Scaling Transformer Inferencearxiv.org
- BrainTransformers: SNN-LLMarxiv.org
- [2106.09685] LoRA: Low-Rank Adaptation of Large Language Modelsarxiv.org
- [2101.03961] Switch Transformers: Scaling to Trillion Parameter Models with Simple and Efficient Sparsityarxiv.org
- https://aclanthology.org/P19-1285.pdfaclanthology.org
- 1706.03762arxiv.org
- More Efficient In-Context Learning with GLaMblog.research.google