flâneur

Mistral 7B | Mistral AI | Open source models

mistral.ai · 844 words · saved by 1 readers

The best 7B model to date, Apache 2.0

Research Mistral 7B September 27, 2023 By Mistral AI team Back to Blog 5 min read Share this post Copy url to clipboard Copied Mistral AI team is proud to release Mistral 7B, the most powerful language model for its size to date. Mistral 7B in short Mistral 7B is a 7.3B parameter model that: Outperforms Llama 2 13B on all benchmarks Outperforms Llama 1 34B on many benchmarks Approaches CodeLlama 7B performance on code, while remaining good at English tasks Uses Grouped-query attention (GQA) for faster inference Uses Sliding Window Attention (SWA) to handle longer sequences at smaller cost We'r

related reading