Don't Build an RL Environment Startup
The first person who sold an RL environment to a frontier AI lab must have felt like they discovered an infinite money glitch. It's no longer a secret that frontier AI labs regularly pay hundreds of thousands, and sometimes millions, for clones of Linear and Salesforce. If you're reading this, you've probably thought about quitting your day job and starting a company that builds these unusually lucrative Next.js apps. In this post, I'll argue that you should hesitate before hopping on the bandwagon. For those unfamiliar, an RL (reinforcement learning) environment is like a sandbox for AI models like Claude and GPT to learn from. It keeps track of an internal state, prompts the AI to take actions to complete a task, and assigns a score based on the outcome. The most obvious kind is a clone of a popular website or enterprise software tool like Doordash, Linear, or Amazon, which teaches the AI to click around and order pizza. It can also be text-only, like the TextArena project, which tea
--> Don't Build an RL Environment Startup Don't Build an RL Environment Startup Don't sell blood to vampires. Posted Sep 7, 2025 by Benjamin Anderson The first person who sold an RL environment to a frontier AI lab must have felt like they discovered an infinite money glitch. It's no longer a secret that frontier AI labs regularly pay hundreds of thousands, and sometimes millions, for clones of Linear and Salesforce. If you're reading this, you've probably thought about quitting your day job and starting a company that builds these unusually lucrative Next.js apps. In this post, I'll argue tha
Explore this link on the map →related reading
- What if RL Environments Aren't Mispriced?benanderson.work
- The Bitter Lesson - RL Environments Versionseancai.com
- RL Environments and RL for Science: Data Foundries and Multi-Agent Architecturesnewsletter.semianalysis.com
- A World of Verifiable Domainsseancai.com
- Trust me bro, just one more RL scale up, this one will be the real scale up with the good environments, the actually legit one, trust me bro — AI Alignment Forumalignmentforum.org
- Pavlov's List: A List of RL Environment Startupspavlovslist.com
- A Taxonomy of RL Environments for LLM Agentsleehanchung.github.io
- Cheap RL tasks will waste compute | Mechanize, Inc.mechanize.work
- Akash Bajwa on X: "RL Environments with Scale AI" / Xx.com
- Introducing OpenReward | General Reasoninggr.inc
- RLHF: Reinforcement Learning from Human Feedbackhuyenchip.com
- Andrej Karpathy — AGI is still a decade awaydwarkesh.com