Mistral Mastery: Fine-Tuning & Fast Inference Guide | by Parikshit Saikia | Oct, 2023 | Medium
medium.com · 1,818 words · saved by 1 readers
Unlocking full potential of Mistral-7B-Inst with QLoRA and vLLM
8 min read Oct 29, 2023 -- I call this “Mistral Pixlipse”, generated in my potato laptop. Mistral-7b-Inst is a game-changer LLM developed by Mistral AI which outperforms many popular LLMs. It’s released under Apache 2.0 license, which makes it suitable to use in a commercial setting. In this article, we will explore how to finetune Mistral-7b-Inst on a custom dataset. We will focus on efficient use of computation resources and fast inference to make it production-ready. This article has two major segments: Fine-Tuning Mistral-7B-Inst with QLoRA Superfast Inference with vLLM…
related reading
- Mediumadithyask.medium.com
- mistralai/Mistral-7B-Instruct-v0.2 · Hugging Facehuggingface.co
- Mistral 7Bmistral.ai
- LoRA Without Regret - Thinking Machines Labthinkingmachines.ai
- Papers Explained 64: Mistralmedium.com
- Efficient LLM Finetuning with Unsloth | Modal Docsmodal.com
- Unsloth update: Mistral support + moreunsloth.ai
- Finetune LLMs on your own consumer hardware using tools from PyTorch and Hugging Face ecosystem – PyTorchpytorch.org
- Fine-tuning LLMs Guide | Unsloth Documentationdocs.unsloth.ai
- 2305.14314arxiv.org
- Fine-Tuning Llama-2: Tailoring Models to Unique Applicationsanyscale.com
- Optimizing inference · Hugging Facehuggingface.co