Sweatshop data is over | Mechanize Inc.

Cheap contractor-labeled data is no longer enough. Future AI progress depends on RL environments built by full-time domain experts.

Tamay Besiroglu, Matthew Barnett, Ege Erdil July 10, 2025 High-quality data is the fuel that drives AI progress, but our approach to AI data needs rethinking. In the past, it was usually sufficient to hire third-party contractors to create datasets for basic text, visual, and audio tasks. This typically involved monotonous, narrowly-scoped labeling and generation tasks performed en masse by low-skill workers, often paid just a few dollars per hour. This “sweatshop data” enabled friendly chatbots, art generators, speech-to-text software, and so on. Back then, sweatshop data was enough…

saved by

• Ratan Kaliani
• Information Flow
• vain glory
• Arjun Khandelwal
• Florent Tavernier

related reading

• Training Imperativesdan.io
• How to fully automate software engineering | Mechanize, Inc.mechanize.work
• Cheap RL tasks will waste compute | Mechanize, Inc.mechanize.work
• The upcoming GPT-3 moment for RL | Mechanize, Inc.mechanize.work
• The data black hole at the center of AIdwarkesh.com
• What if RL Environments Aren't Mispriced?benanderson.work
• There Are No New Ideas in AI… Only New Datasetsblog.jxmo.io
• Oversight Assistants: Turning Compute into Understandingbounded-regret.ghost.io
• Trust me bro, just one more RL scale up, this one will be the real scale up with the good environments, the actually legit one, trust me bro — AI Alignment Forumalignmentforum.org
• RL Environments and RL for Science: Data Foundries and Multi-Agent Architecturesnewsletter.semianalysis.com
• After Automation | Everyevery.to
• Thoughts on AI progress (Dec 2025)substack.com