Robust Regression for Safe Exploration in Control
We study the problem of safe learning and exploration in sequential control problems. The goal is to safely collect data samples from operating in an environment, in order to learn to achieve a challenging control goal (e.g., an agile maneuver close to a boundary). A central challenge in this setting is how to quantify uncertainty in order to choose provably-safe actions that allow us to collect informative data and reduce uncertainty, thereby achieving both improved controller safety and optimality. To address this challenge, we present a deep robust regression model that is trained to directly predict the uncertainty bounds for safe exploration. We derive generalization bounds for learning, and connect them with safety and stability bounds in control. We demonstrate empirically that our robust regression approach can outperform conventional Gaussian process (GP) based safe exploration in settings where it is difficult to specify a good GP prior. A key challenge in data-driven design
Abstract We study the problem of safe learning and exploration in sequential control problems. The goal is to safely collect data samples from operating in an environment, in order to learn to achieve a challenging control goal (e.g., an agile maneuver close to a boundary). A central challenge in this setting is how to quantify uncertainty in order to choose provably-safe actions that allow us to collect informative data and reduce uncertainty, thereby achieving both improved controller safety and optimality. To address this challenge, we present a deep robust regression model that is…
saved by
related reading
- A Free Lunch from the Noise:Provable and Practical Exploration for Representation Learningarxiv.org
- Emergent Dexterity via Diverse Resets and Large-Scale Reinforcement Learningarxiv.org
- Contrastive Learning as Goal-Conditioned Reinforcement Learningarxiv.org
- [2605.01172] A Theory of Generalization in Deep Learningarxiv.org
- [1707.06347] Proximal Policy Optimization Algorithmsarxiv.org
- [1709.06560] Deep Reinforcement Learning that Mattersarxiv.org
- Robotic World Model: A Neural Network Simulator for Robust Policy Optimization in Roboticsarxiv.org
- A Unifying Perspective for Safety of Stochastic Systems: From Barrier Functions to Finite Abstractionsarxiv.org
- Exploration for the Efficient Deployment of Reinforcement Learning Agentsopenreview.net
- 1011.0686arxiv.org
- 2208.05129arxiv.org
- [1805.12114] Deep Reinforcement Learning in a Handful of Trials using Probabilistic Dynamics Modelsar5iv.labs.arxiv.org