sapientinc/HRM-Text: HRM-Text is a 1B text generation model based on the HRM architecture, strengthened by task completion and latent space reasoning. ·
github.com · 1,465 words · saved by 1 readers
HRM-Text is a 1B text generation model based on the HRM architecture, strengthened by task completion and latent space reasoning.
HRM-Text: Efficient Pretraining Beyond Scaling 🌟 Pretrain a foundation model from scratch with ~$1000. 🌠 HRM-Text is a 1B text generation model based on the HRM architecture, strengthened by task completion and latent space reasoning. It offers a full pretraining framework, making foundation model pretraining accessible with 130-600x less compute and 150-900x less data. It is built upon a hierarchical recurrent architecture, PrefixLM sequence packing, FlashAttention 3 kernels, PyTorch FSDP2 training, evaluation, and checkpoint conversion tooling. Join 1200+ HRM Developers on Our Discord…
saved by
related reading
- Hugging Face – The AI community building the future.huggingface.co
- Parsed | Custom, interpretable AI systems that continuously learnparsed.com
- AI Model & API Providers Analysis | Artificial Analysisartificialanalysis.ai
- Replicate - Run AI with an APIreplicate.com
- PostTrainBenchposttrainbench.com
- Datacurve | The data engine for frontier AIdatacurve.ai
- GitHub - brexhq/prompt-engineering: Tips and tricks for working with Large Language Models like OpenAI's GPT-4.github.com
- GitHub - rasbt/LLMs-from-scratch: Implement a ChatGPT-like LLM in PyTorch from scratch, step by stepgithub.com
- Compare AI Models: Pricing, Context & Benchmarks | OpenRouteropenrouter.ai
- Llama 2 · Hugging Facehuggingface.co
- Tinkerthinkingmachines.ai
- Branches · HazyResearch/intelligence-per-watt · GitHubgithub.com