Levanter
You could not prevent a thunderstorm, but you could use the electricity; you could not direct the wind, but you could trim your sail so as to propel your vessel as you pleased, no matter which way the wind blew. — Cora L. V. Hatch Levanter is a framework for training large language models (LLMs) and other foundation models that strives for legibility, scalability, and reproducibility: We built Levanter with JAX, Equinox, and Haliax. Levanter's documentation is available at levanter.readthedocs.io. Haliax's documentation is available at haliax.readthedocs.io. The code is released on GitHub: Levanter repository. To get started, please refer to the User Guide's chapters: Please also see the guides in the menu on the left. To contribute, please refer to the Contributing Guide.
Home You could not prevent a thunderstorm, but you could use the electricity; you could not direct the wind, but you could trim your sail so as to propel your vessel as you pleased, no matter which way the wind blew. — Cora L. V. Hatch Levanter is a framework for training large language models (LLMs) and other foundation models that strives for legibility, scalability, and reproducibility: Legible: Levanter uses our named tensor library Haliax to write easy-to-follow, composable deep learning code, while still being high performance. Scalable: Levanter scales to large models, and to be…
saved by
related reading
- GitHub - marin-community/haliax: Named Tensors for Legible Deep Learning in JAX · GitHubgithub.com
- Tinkerthinkingmachines.ai
- Let's build GPT: from scratch, in code, spelled out.youtube.com
- [2106.09685] LoRA: Low-Rank Adaptation of Large Language Modelsarxiv.org
- GitHub - rasbt/LLMs-from-scratch: Implement a ChatGPT-like LLM in PyTorch from scratch, step by stepgithub.com
- Mamba: The Easy Wayjackcook.com
- Mosaic LLMs: GPT-3 quality formosaicml.com
- Build a Transformer in JAX from scratch: how to write and train your own models | AI Summertheaisummer.com
- The Llama Hitchiking Guide to Local LLMs – hackerllamaosanseviero.github.io
- GitHub - linkedin/Liger-Kernel: Efficient Triton Kernels for LLM Traininggithub.com
- Productizing Large Language Modelsblog.replit.com
- Llama 2 · Hugging Facehuggingface.co