AI Capabilities Can Be Significantly Improved Without Expensive Retraining | Epoch AI
While scaling compute is key to improving LLMs, post-training enhancements can offer gains equivalent to 5-20x more compute at less than 1% of the cost.
AI capabilities can be significantly improved without expensive retraining | Epoch AI The massive computation used to train LLMs and similar foundation models has been one of the main drivers of AI progress in recent years, which has led to the recognition of the “Bitter Lesson”: that general methods that better leverage computational power are ultimately the most effective ( Sutton, 2019 ). The cost of training frontier models has now become so high that only a handful of actors can afford it ( Epoch AI, 2023 ). Our study explores methods of improving performance after training that don’t rel
Explore this link on the map →related reading
- AI progress is about to speed up | Epoch AIepoch.ai
- AI in 2025: gestalt — LessWronglesswrong.com
- Composer2.pdfcursor.com
- I. From GPT-4 to AGI: Counting the OOMs - SITUATIONAL AWARENESSsituational-awareness.ai
- My picture of the present in AI — LessWronglesswrong.com
- PostTrainBenchposttrainbench.com
- Noam Brown on X: "Implications of Large-Scale Test-Time Compute" / Xx.com
- How fast is AI improving? - AI Digesttheaidigest.org
- Andrej Karpathy — AGI is still a decade awaydwarkesh.com
- Dario Amodei — "We are near the end of the exponential"dwarkesh.com
- Elicitation, the simplest way to understand post-traininginterconnects.ai
- PostTrainBench: Measuring AI Ability to Perform LLM Post-Trainingaisagroup.substack.com