How not to do research - Rajan Agarwal
rajan.sh · 2,796 words · saved by 8 readers
Lessons learned from building multiplayer world models. Built a video tokenizer with spatial attention and a dynamics model with action spaces.
How not to do research - Rajan Agarwal I don't usually share when things go wrong. Like most, my public work tends to be the stuff that worked, but I learned a lot from this project, and I want to share what I learned about the problem, and how not to do research. I tried to train a Genie-style world model that could learn to segment two players and their independent action spaces from Pong video alone, without labels or hardcoded structure. The model kept collapsing to degenerate solutions. It would always track the ball as "player 1" or even treat the score as an agent. I eventually realized
saved by
- Tasha Pais
- Benedict Neo
- Yudhister Joel Kumar
- Rohan Kanti
- Ishan Mukherjee
- Vihan Tiwari
- Aaron Liu
- Nathan Chen
related reading
- Debugging Reinforcement Learning Systemsandyljones.com
- The First Fully General Computer Action Model | blogsi.inc
- LLMs can invent their own compression - Rajan Agarwalrajan.sh
- Tiny Interaction Models - Rajan Agarwalrajan.sh
- Learning Beyond Gradientstrinkle23897.github.io
- World Models | Rohit Bandarurohitbandaru.github.io
- How to train a frontier-level world modelnext-state.github.io
- Natural Deception with RL - Rajan Agarwalrajan.sh
- Deep Reinforcement Learning: Pong from Pixelskarpathy.github.io
- [2206.11795] Video PreTraining (VPT): Learning to Act by Watching Unlabeled Online Videosarxiv.org
- pdfopenreview.net
- Rajan Agarwalrajan.sh