huggingface/nanotron: Minimalistic large language model 3D-parallelism training ·

Minimalistic large language model 3D-parallelism training

⚡️ Nanotron Installation • Quick Start • Features • Benchmarks • Contributing Pretraining models made easy Nanotron is a library for pretraining transformer models. It provides a simple and flexible API to pretrain models on custom datasets. Nanotron is designed to be easy to use, fast, and scalable. It is built with the following principles in mind: Simplicity : Nanotron is designed to be easy to use. It provides a simple and flexible API to pretrain models on custom datasets. Performance : Optimized for speed and scalability, Nanotron uses the latest techniques to train models faster and mor

saved by

• Senthilnathan Thirumurugan

related reading

• The Ultra-Scale Playbook - a Hugging Face Space by nanotronhuggingface.co
• The Ultra-Scale Playbook - a Hugging Face Space by nanotronhuggingface.co
• Predict Memory - a Hugging Face Space by nanotronhuggingface.co
• GitHub - karpathy/nanochat: The best ChatGPT that $100 can buy.github.com
• GitHub - karpathy/autoresearch: AI agents running research on single-GPU nanochat training automaticallygithub.com
• Let's build GPT: from scratch, in code, spelled out.youtube.com
• microgptkarpathy.github.io
• GitHub - rasbt/LLMs-from-scratch: Implement a ChatGPT-like LLM in PyTorch from scratch, step by stepgithub.com
• GitHub - NVIDIA-NeMo/Automodel: 🚀 Pytorch Distributed native training library for LLMs/VLMs with OOTB Hugging Face supportgithub.com
• Llama 2 · Hugging Facehuggingface.co
• ⭐️ Fast LLM Inference From Scratchandrewkchan.dev
• Let's reproduce GPT-2 (124M)youtube.com