MLC LLM | Home

MLC LLM is a universal solution that allows any language model to be deployed natively on a diverse set of hardware backends and native applications, plus a productive framework for everyone to further optimize model performance for their own use cases. Everything runs locally with no server support and accelerated with local GPUs on your phone and laptop. Check out our GitHub repository to see how we did it. You can also read through instructions below for trying out demos. This section contains the instructions to run large-language models and chatbot natively on your environment. Try out this TestFlight page (limited to the first 9000 users) to install and use our example iOS chat app built for iPhone. Our app itself needs about 4GB of memory to run. Considering the iOS and other running applications, we will need a recent iPhone with 6GB (or more) of memory to run the app. We only tested the application on iPhone 14 Pro Max and iPhone 12 Pro. You can also check out our GitHub repo

saved by

• James Webb