Pre-training isn't dead, it’s just resting

For two years after GPT-4, every model OpenAI released was smaller. On February 27, 2025, that finally changed. OpenAI launched GPT-4.5, calling it “the next step in scaling the unsupervised learning paradigm”. Here’s what people thought: The launch video didn’t do the model any favours. Rather than focusing on GPT-4.5’s strengths, it highlighted how small the improvements were relative to GPT-4o and that it underperformed o3-mini, a 50x cheaper model. Only a month and a half after the model’s release, they announced it was going to be deprecated. When GPT-4 was released in 2023, it was seen as something fundamentally unprecedented. It blew past GPT-3.5 on the majority of benchmarks, and in some cases, saturated them entirely by reaching human-level performance. GPT-4’s launch also came with a developer livestream full of feats that seemed unimaginable for GPT-3.5, such as converting a hand-drawn wireframe into a fully functioning website in a single request. Where GPT-4 felt like a st

For two years after GPT-4, every model OpenAI released was smaller. On February 27, 2025, that finally changed. OpenAI launched GPT-4.5, calling it “the next step in scaling the unsupervised learning paradigm”. Here’s what people thought: The launch video didn’t do the model any favours. Rather than focusing on GPT-4.5’s strengths, it highlighted how small the improvements were relative to GPT-4o and that it underperformed o3-mini, a 50x cheaper model. Only a month and a half after the model’s release, they announced it was going to be deprecated. When GPT-4 was released in 2023, it was seen a