If it makes you feel any better, I've been doing this for a while and it took me... | Hacker News
Also, what we know about good CNN design from supervised learning land doesn't seem to apply to reinforcement learning land, because you're mostly bottlenecked by credit assignment / supervision bitrate, not by a lack of a powerful representation. Your ResNets, batchnorms, or very deep networks have no power here. SL wants to work. Even if you screw something up you'll usually get something non-random back. RL must be forced to work. If you screw something up or don't tune something well enough you're exceedingly likely to get a policy that is even worse than random. And even if it's all well tuned you'll get a bad policy 30% of the time, just because. Long story short your failure is more due to the difficulty of deep RL, and much less due to the difficulty of "designing neural networks". RL is about learning expected-reward-maximizing policies for environments that you get to interact with. Common benchmarks currently mostly include games (e.g. ATARI, AlphaGo, VizDoom), physics-based
If it makes you feel any better, I've been doing this for a while and it took me... | Hacker News Hacker News new | past | comments | ask | show | jobs | submit login karpathy on Jan 30, 2017 | parent | context | favorite | on: Outrageously Large Neural Networks: The Sparsely-G... If it makes you feel any better, I've been doing this for a while and it took me last ~6 weeks to get a from-scratch policy gradients implementation to work 50% of the time on a bunch of RL problems. And I also have a GPU cluster available to me, and a number of friends I get lunch with every day who've been in the a
Explore this link on the map →related reading
- Part 1: Key Concepts in RL - Spinning Up documentationspinningup.openai.com
- Debugging Reinforcement Learning Systemsandyljones.com
- Deep Reinforcement Learning Doesn't Work Yetalexirpan.com
- Just Ask for Generalization | Eric Jangevjang.com
- A (Long) Peek into Reinforcement Learning | Lil'Loglilianweng.github.io
- Deep Reinforcement Learning: Pong from Pixelskarpathy.github.io
- Andrej Karpathy — AGI is still a decade awaydwarkesh.com
- Reinforcement learning - Wikipediaen.wikipedia.org
- Deep Q-Networks Explained — LessWronglesswrong.com
- Evolution as Backstop for Reinforcement Learning · Gwern.netgwern.net
- Reward is not the optimization target — LessWronglesswrong.com
- RLHF: Reinforcement Learning from Human Feedbackhuyenchip.com