[2403.00579] NeuPIMs: NPU-PIM Heterogeneous Acceleration for Batched LLM Inferencing

arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website. Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them. Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs. arXiv Operational Status Get status notifications via email or slack

[2403.00579] NeuPIMs: NPU-PIM Heterogeneous Acceleration for Batched LLM Inferencing --> Computer Science > Hardware Architecture arXiv:2403.00579 (cs) [Submitted on 1 Mar 2024 ( v1 ), last revised 29 Mar 2024 (this version, v3)] Title: NeuPIMs: NPU-PIM Heterogeneous Acceleration for Batched LLM Inferencing Authors: Guseul Heo , Sangyeop Lee , Jaehong Cho , Hyunmin Choi , Sanghyeon Lee , Hyungkyu Ham , Gwangsun Kim , Divya Mahajan , Jongse Park View a PDF of the paper titled NeuPIMs: NPU-PIM Heterogeneous Acceleration for Batched LLM Inferencing, by Guseul Heo and 8 other authors View PDF Abst

saved by

• Pavel

related reading

• [2312.15159] Understanding the Potential of FPGA-Based Spatial Acceleration for Large Language Model Inferencearxiv.org
• [2403.06664] Smart-Infinity: Fast Large Language Model Training using Near-Storage Processing on a Real Systemarxiv.org
• AttentionLego: An Open-Source Building Block For Spatially-Scalable Large Language Model Accelerator With Processing-In-Memory Technologyarxiv.org
• Chameleon: a Heterogeneous and Disaggregated Accelerator System for Retrieval-Augmented Language Modelsarxiv.org
• [2304.07493] OliVe: Accelerating Large Language Models via Hardware-friendly Outlier-Victim Pair Quantizationarxiv.org
• Quantization and Hardware Architecture Co-Design for Matrix-Vector Multiplications of Large Language Modelsieeexplore.ieee.org
• A$^3$: Accelerating Attention Mechanisms in Neural Networks with Approximationarxiv.org
• [2209.10797] DFX: A Low-latency Multi-FPGA Appliance for Accelerating Transformer-based Text Generationarxiv.org
• [2011.14203] EdgeBERT: Sentence-Level Energy Optimizations for Latency-Aware Multi-Task NLP Inferencearxiv.org
• ⭐️ Fast LLM Inference From Scratchandrewkchan.dev
• HeteroLLM: Accelerating Large Language Model Inference on Mobile SoCs with Heterogeneous AI Acceleratorsarxiv.org
• 2502.11089arxiv.org