AWQ: Activation-aware Weight Quantization for On-Device LLM Compression and Acceleration
arxiv.org · 6,251 words · saved by 1 readers
N/A
AWQ: A CTIVATION - AWARE W EIGHT Q UANTIZATION FOR O N -D EVICE LLM C OMPRESSION AND ACCELERATION Ji Lin * 1 Jiaming Tang * 1 2 Haotian Tang † 1 Shang Yang † 1 Wei-Ming Chen 3 Wei-Chen Wang 1 Guangxuan Xiao 1 Xingyu Dang 1 4 Chuang Gan 5 6 Song Han 1 3 https://github.com/mit-han-lab/llm-awq…
related reading
- Quantization from the ground upngrok.com
- 2305.14314arxiv.org
- What Makes Low-Bit Quantization-Aware Training Work for Reasoning LLMs? A Systematic Studyarxiv.org
- [2510.11696] QeRL: Beyond Efficiency -- Quantization-enhanced Reinforcement Learning for LLMsarxiv.org
- SmoothQuant: Accurate and EfficientPost-Training Quantization for Large Language Modelsarxiv.org
- [2510.13786] The Art of Scaling Reinforcement Learning Compute for LLMsarxiv.org
- A Guide to Quantization in LLMs | Symbl.aisymbl.ai
- LLMLingua: Compressing Prompts for Accelerated Inference of Large Language Modelsarxiv.org
- 2408.14690arxiv.org
- 2502.11089arxiv.org
- Efficient LLM inferencefinbarrtimbers.substack.com
- GPTQ: Accurate Post-Training Quantization for Generative Pre-Trained Transformersarxiv.org