[2209.10797] DFX: A Low-latency Multi-FPGA Appliance for Accelerating Transformer-based Text Generation

arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website. Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them. Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs. arXiv Operational Status Get status notifications via email or slack

[2209.10797] DFX: A Low-latency Multi-FPGA Appliance for Accelerating Transformer-based Text Generation --> Electrical Engineering and Systems Science > Systems and Control arXiv:2209.10797 (eess) [Submitted on 22 Sep 2022] Title: DFX: A Low-latency Multi-FPGA Appliance for Accelerating Transformer-based Text Generation Authors: Seongmin Hong , Seungjae Moon , Junsoo Kim , Sungjae Lee , Minsub Kim , Dongsoo Lee , Joo-Young Kim View a PDF of the paper titled DFX: A Low-latency Multi-FPGA Appliance for Accelerating Transformer-based Text Generation, by Seongmin Hong and 6 other authors View PDF

saved by

• Pavel

related reading

• [2312.15159] Understanding the Potential of FPGA-Based Spatial Acceleration for Large Language Model Inferencearxiv.org
• FTRANS: Energy-Efficient Acceleration of Transformers using FPGAarxiv.org
• [2403.06664] Smart-Infinity: Fast Large Language Model Training using Near-Storage Processing on a Real Systemarxiv.org
• [2403.00579] NeuPIMs: NPU-PIM Heterogeneous Acceleration for Batched LLM Inferencingarxiv.org
• [2201.06618] VAQF: Fully Automatic Software-Hardware Co-Design Framework for Low-Bit Vision Transformerarxiv.org
• Chameleon: a Heterogeneous and Disaggregated Accelerator System for Retrieval-Augmented Language Modelsarxiv.org
• [2011.14203] EdgeBERT: Sentence-Level Energy Optimizations for Latency-Aware Multi-Task NLP Inferencearxiv.org
• Hardware Accelerator for Multi-Head Attention and Position-Wise Feed-Forward in the Transformerieeexplore.ieee.org
• A$^3$: Accelerating Attention Mechanisms in Neural Networks with Approximationarxiv.org
• AttentionLego: An Open-Source Building Block For Spatially-Scalable Large Language Model Accelerator With Processing-In-Memory Technologyarxiv.org
• [2207.00032] DeepSpeed Inference: Enabling Efficient Inference of Transformer Models at Unprecedented Scalearxiv.org
• [2401.05566] Sleeper Agents: Training Deceptive LLMs that Persist Through Safety Trainingarxiv.org