flâneur

The Training Imperative

sdan.io · 1,025 words · saved by 5 readers

Most AI startups today build on top of the lab APIs. That works for now. But if history rhymes, every serious AI company will eventually train its own models — even the wrappers. Why? Because the barrier to doing so is collapsing. Distillation, fine-tuning, and post-training get easier every month. By the time ASI-level players are soaking up all the capital and talent, the only way to stay relevant will be to own your own models. We started with research labs training LLMs. Then came Midjourney and Stable Diffusion, building on diffusion research. Then everything took off and the labs started serious commercialization in late 2022 and early 2023. By 2024, anyone with a rack of GPUs could launch a training run. By 2025, DeepSeek distilled a frontier model for $6M that hit o1-level reasoning — just four months after OpenAI announced o1, and two months after release. The point: what’s behind the API isn’t far off from what’s available to everyone else. The ingredients are simple: data, c

Most AI startups today build on top of the lab APIs. That works for now. But if history rhymes, every serious AI company will eventually train its own models — even the wrappers. Why? Because the barrier to doing so is collapsing. Distillation, fine-tuning, and post-training get easier every month. By the time ASI-level players are soaking up all the capital and talent, the only way to stay relevant will be to own your own models. How We Got Here In 2019, OpenAI released GPT-2 in staged checkpoints (by size) due to safety concerns. After GPT-3, others—notably Cohere and Primer—began training a

saved by

related reading