Semi-supervised reinforcement learning | by Paul Christiano | AI Alignment

As usual, our goal is to quickly learn a policy which receives a high reward per episode. There are two natural flavors of semi-supervised RL: We can apply a traditional RL algorithm to the semi-supervised setting by simply ignoring all of the unlabelled episodes. This will generally result in very slow learning. The interesting challenge is to learn efficiently from the unlabelled episodes. I think that semi-supervised RL is a valuable ingredient for AI control, as well as an interesting research problem in reinforcement learning. As a simple example, consider an RL system which learns from the user pressing a “reward button” — each time the agent performs a task well the user presses the button to let it know. (A realistic design would more likely use verbal approval, more subtle cues, or performance measures that don’t involve the user at all. But a very simple example makes the point clear.) If our system is a competent RL agent maximizing button presses, it will eventually learn t

As usual, our goal is to quickly learn a policy which receives a high reward per episode. There are two natural flavors of semi-supervised RL: We can apply a traditional RL algorithm to the semi-supervised setting by simply ignoring all of the unlabelled episodes. This will generally result in very slow learning. The interesting challenge is to learn efficiently from the unlabelled episodes. I think that semi-supervised RL is a valuable ingredient for AI control, as well as an interesting research problem in reinforcement learning. As a simple example, consider an RL system which learns from t