The Unreasonable Effectiveness of Easy Training Data for Hard Tasks
Week 2 of Superslow and so far it’s living up to its namesake 🙂 This week, we’re exploring a paper that examines how effective “easy” training data can be in developing strong model capabilities compared to training with more challenging or “hard” data. This paper is authored by one of my best friends, Peter Hase, who recently wrapped up a residency on the safety team at Anthropic. Hope you enjoy! Let’s dive in! Thanks for reading! Hit subscribe to stay updated on the most interesting news in AI. At a high level, this paper aims to answer the question: if I train my models on “easy” data, can I get the same performance as training on “hard” data? Hase et. al measure four things in this paper: How do we know if data is “hard” vs “easy”? Do “easy”-trained models score highly on “hard” tests? What are the tradeoffs between collecting easy vs hard datasets? If the final upshot is that “easy”-trained models perform similarly well to “hard-trained models on hard tests, is this behavior cons
The Unreasonable Effectiveness of Easy Training Data for Hard Tasks 0 Superslow AI Newsletter Posts The Unreasonable Effectiveness of Easy Training Data for Hard Tasks The Unreasonable Effectiveness of Easy Training Data for Hard Tasks Alexandra Barr June 16, 2025 Hi everyone 👋 Week 2 of Superslow and so far it’s living up to its namesake 🙂 This week, we’re exploring a paper that examines how effective “easy” training data can be in developing strong model capabilities compared to training with more challenging or “hard” data. This paper is authored by one of my best friends, Peter Hase, who
saved by
related reading
- Automated Weak-to-Strong Researcheralignment.anthropic.com
- Without specific countermeasures, the easiest path to transformative AI likely leads to AI takeover — LessWronglesswrong.com
- [2604.22082] Removing Sandbagging in LLMs by Training with Weak Supervisionarxiv.org
- 3 Challenges and 2 Hopes for the Safety of Unsupervised Elicitationalignment.anthropic.com
- Scaling is subtler than it seemsberen.io
- Constitutional AI: Harmlessness from AI Feedbackarxiv.org
- Narrow Misalignment is Hard, Emergent Misalignment is Easy — LessWronglesswrong.com
- Simulated Users & Sad LLMs1a3orn.com
- Maybe I was too harsh on deep learning theory (three days ago) — LessWronglesswrong.com
- How to Explore to Scale RL Training of LLMs on Hard Problems? – Machine Learning Blog | ML@CMU | Carnegie Mellon Universityblog.ml.cmu.edu
- IsoCompute Playbook: Optimally Scaling Sampling Compute for RL Training of LLMscompute-optimal-rl-llm-scaling.github.io
- Is AI Rare or Everywhere? · Hazy Researchhazyresearch.stanford.edu