Mistral Mastery: Fine-Tuning & Fast Inference Guide | by Parikshit Saikia | Oct, 2023 | Medium
medium.com · 1,818 words · saved by 1 readers
Unlocking full potential of Mistral-7B-Inst with QLoRA and vLLM
8 min read Oct 29, 2023 -- I call this “Mistral Pixlipse”, generated in my potato laptop. Mistral-7b-Inst is a game-changer LLM developed by Mistral AI which outperforms many popular LLMs. It’s released under Apache 2.0 license, which makes it suitable to use in a commercial setting. In this article, we will explore how to finetune Mistral-7b-Inst on a custom dataset. We will focus on efficient use of computation resources and fast inference to make it production-ready. This article has two major segments: Fine-Tuning Mistral-7B-Inst with QLoRA Superfast Inference with vLLM…
related reading
- Mediumadithyask.medium.com
- mistralai/Mistral-7B-Instruct-v0.2 · Hugging Facehuggingface.co
- Mistral 7Bmistral.ai
- LoRA Without Regret - Thinking Machines Labthinkingmachines.ai
- Papers Explained 64: Mistralmedium.com
- TheBloke/Mistral-7B-Instruct-v0.1-AWQ · Hugging Facehuggingface.co
- Unsloth update: Mistral support + moreunsloth.ai
- LLM Inference Performance Engineering: Best Practices | Databricks Blogdatabricks.com
- Finetune LLMs on your own consumer hardware using tools from PyTorch and Hugging Face ecosystem – PyTorchpytorch.org
- Welcome Mixtral - a SOTA Mixture of Experts on Hugging Facehuggingface.co
- Efficient LLM Finetuning with Unsloth | Modal Docsmodal.com
- microgptkarpathy.github.io