openai/parameter-golf: Train the smallest LM you can that fits in 16MB. Best model wins! ·
github.com · 3,968 words · saved by 1 readers
Train the smallest LM you can that fits in 16MB. Best model wins!
OpenAI Model Craft Challenge: Parameter Golf is a challenge to train the best language model that fits in a 16MB artifact and trains in under 10 minutes on 8xH100s, evaluated by compression on the FineWeb validation set (tokenizer-agnostic, bits per byte). This challenge is heavily inspired by the NanoGPT Speedrunning challenge, where participants compete to train a model that reaches 3.28 FineWeb validation loss as quickly as possible. We're excited to see how optimizing for a parameter-constrained setting pushes people toward unique architectures (test-time compute, aggressive parameter…
saved by
related reading
- Keller Jordan (@kellerjordan0) on Xx.com
- Parameter Golf Research Gardengolf.agustif.com
- Composer2.pdfcursor.com
- Pathways Language Model (PaLM): Scaling to 540 Billion Parameters for Breakthrouai.googleblog.com
- Quantization from the ground upngrok.com
- PostTrainBenchposttrainbench.com
- GitHub - karpathy/nanochat: The best ChatGPT that $100 can buy.github.com
- Tinkerthinkingmachines.ai
- Hugging Face – The AI community building the future.huggingface.co
- Chinchillaarxiv.org
- 2305.14314arxiv.org
- [Jan 7 2026] nanochat miniseries v1 · karpathy nanochat · Discussion #420github.com