TheBloke/Mistral-7B-Instruct-v0.1-AWQ · Hugging Face
AWQ is an efficient, accurate and blazing-fast low-bit weight quantization method, currently supporting 4-bit quantization. Compared to GPTQ, it offers faster Transformers-based inference.
Mistral 7B Instruct v0.1 - AWQ Model creator: Mistral AI Original model: Mistral 7B Instruct v0.1 Description This repo contains AWQ model files for Mistral AI's Mistral 7B Instruct v0.1. About AWQ AWQ is an efficient, accurate and blazing-fast low-bit weight quantization method, currently supporting 4-bit quantization. Compared to GPTQ, it offers faster Transformers-based inference. Mistral AWQs These are experimental first AWQs for the brand-new model format, Mistral. As of September 29th 2023, they are only supported by AutoAWQ (version 0.1.1+) Repositories available AWQ…
related reading
- mistralai/Mistral-7B-Instruct-v0.2 · Hugging Facehuggingface.co
- Mistral Mastery: Fine-Tuning & Fast Inference Guidemedium.com
- microgptkarpathy.github.io
- Inkling: Our Open-Weights Model - Thinking Machines Labthinkingmachines.ai
- Welcome Mixtral - a SOTA Mixture of Experts on Hugging Facehuggingface.co
- Mistral 7Bmistral.ai
- Papers Explained 64: Mistralmedium.com
- Uncensored Modelserichartford.com
- Llama 2 · Hugging Facehuggingface.co
- Quantization · Hugging Facehuggingface.co
- GitHub - NVIDIA-NeMo/Automodel: 🚀 Pytorch Distributed native training library for LLMs/VLMs with OOTB Hugging Face supportgithub.com
- Qwen3: Think Deeper, Act Faster | Qwenqwenlm.github.io