Unsloth update: Mistral support + more

unsloth About Pricing Blog Contact Get started Blog Finetune Mistral 14x faster Dec 14, 2023 • By Daniel Han Unsloth open source: Mistral 7B 1xA100 2.2x faster Code Llama 34B 1xA100 1.9x faster Llama 7B 1xA100 2.2x faster Llama 7B 1xT4 2x faster We’re excited to release QLoRA support for Mistral 7B, CodeLlama 34B, and all other models based on the Llama architecture! We added sliding window attention, preliminary Windows and DPO support, and will share all 59 notebooks for reproducing our numbers. You can now QLoRA finetune Mistral 7B 2.2x on 1x A100 faster with 62% less memory or 12.4GB peak VRAM. CodeLlama 34B is 1.9x faster on 1x A100, using 32% less memory or 27GB peak VRAM. It finally doesn’t OOM! Unsloth Pro version: Mistral 7B 1xA100 14x faster Code Llama 34B 1xA100 13x faster Llama 7B 1xA100 21x faster Llama 7B 2xT4 28x faster Our PRO version can finetune Mistral 7B a whopping 14x faster on 1x A100, using 70% less peak VRAM, and CodeLlama 34B is 13x faster on 1x A100 using 50%

Unsloth update: Mistral support + more unsloth Models Blog Unsloth Studio✨ Docs Blog Finetune Mistral 14x faster Dec 14, 2023 • By Daniel Han Dec 14, 2023 • By Daniel Han Unsloth open source: Mistral 7B 1xA100 2.2x faster Code Llama 34B 1xA100 1.9x faster Llama 7B 1xA100 2.2x faster Llama 7B 1xT4 2x faster We’re excited to release QLoRA support for Mistral 7B, CodeLlama 34B, and all other models based on the Llama architecture! We added sliding window attention, preliminary Windows and DPO support, and will share all 59 notebooks for reproducing our numbers. You can now QLoRA finetune Mistral

related reading

• Memory Efficient RL | Unsloth Documentationdocs.unsloth.ai
• LoRA Without Regret - Thinking Machines Labthinkingmachines.ai
• Mistral Mastery: Fine-Tuning & Fast Inference Guidemedium.com
• Efficient LLM Finetuning with Unsloth | Modal Docsmodal.com
• Mistral 7Bmistral.ai
• Papers Explained 64: Mistralmedium.com
• 2305.14314arxiv.org
• Tutorial: How to Finetune Llama-3 and Use In Ollama | Unsloth Documentationdocs.unsloth.ai
• Fine-tuning LLMs Guide | Unsloth Documentationdocs.unsloth.ai
• ⭐️ Fast LLM Inference From Scratchandrewkchan.dev
• GitHub - NVIDIA-NeMo/Automodel: 🚀 Pytorch Distributed native training library for LLMs/VLMs with OOTB Hugging Face supportgithub.com
• Look Ma, No Bubbles! Designing a Low-Latency Megakernel for Llama-1B · Hazy Researchhazyresearch.stanford.edu