Papers Explained 64: Mistral 7B. Mistral 7B is an LLM engineered for… | by Ritvik Rastogi | DAIR.AI | Oct, 2023 | Medium
medium.com · 3,340 words · saved by 1 readers
Mistral 7B is an LLM engineered for superior performance and efficiency. It leverages grouped-query attention (GQA) for faster inference…
15 min read Oct 23, 2023 -- Press enter or click to view image in full size Mistral 7B is an LLM engineered for superior performance and efficiency. It leverages grouped-query attention (GQA) for faster inference, coupled with sliding window attention (SWA) to effectively handle sequences of arbitrary length with a reduced inference cost. Mistral 7B outperforms the best open 13B model (Llama 2) across all evaluated benchmarks, and the best released 34B model (Llama 1) in reasoning, mathematics, and code generation. Mistral 7B — Instruct, model fine-tuned to follow instructions on…
related reading
- Mistral 7Bmistral.ai
- The Big LLM Architecture Comparisonmagazine.sebastianraschka.com
- LLM Inference Performance Engineering: Best Practices | Databricks Blogdatabricks.com
- Mistral Mastery: Fine-Tuning & Fast Inference Guidemedium.com
- mistralai/Mistral-7B-Instruct-v0.2 · Hugging Facehuggingface.co
- LLM Architecture Gallery | Sebastian Raschka, PhDsebastianraschka.com
- Mamba: The Easy Wayjackcook.com
- Mixtral of Expertsarxiv.org
- Welcome Mixtral - a SOTA Mixture of Experts on Hugging Facehuggingface.co
- ⭐️ Fast LLM Inference From Scratchandrewkchan.dev
- A Visual Guide to Mamba and State Space Modelsnewsletter.maartengrootendorst.com
- Unsloth update: Mistral support + moreunsloth.ai