[2406.06714] Coprocessor Actor Critic: A Model-Based Reinforcement Learning Approach For Adaptive Brain Stimulation

Adaptive brain stimulation can treat neurological conditions such as Parkinson's disease and post-stroke motor deficits by influencing abnormal neural activity. Because of patient heterogeneity, each patient requires a unique stimulation policy to achieve optimal neural responses. Model-free reinforcement learning (MFRL) holds promise in learning effective policies for a variety of similar control tasks, but is limited in domains like brain stimulation by a need for numerous costly environment interactions. In this work we introduce Coprocessor Actor Critic, a novel, model-based reinforcement learning (MBRL) approach for learning neural coprocessor policies for brain stimulation. Our key insight is that coprocessor policy learning is a combination of learning how to act optimally in the world and learning how to induce optimal actions in the world through stimulation of an injured brain. We show that our approach overcomes the limitations of traditional MFRL methods in terms of sample efficiency and task success and outperforms baseline MBRL approaches in a neurologically realistic model of an injured brain.

View PDF HTML (experimental) Abstract:Adaptive brain stimulation can treat neurological conditions such as Parkinson's disease and post-stroke motor deficits by influencing abnormal neural activity. Because of patient heterogeneity, each patient requires a unique stimulation policy to achieve optimal neural responses. Model-free reinforcement learning (MFRL) holds promise in learning effective policies for a variety of similar control tasks, but is limited in domains like brain stimulation by a need for numerous costly environment interactions. In this work we introduce Coprocessor Actor…

saved by

• Amos You

related reading

• [1709.06560] Deep Reinforcement Learning that Mattersarxiv.org
• [2103.06326] S4RL: Surprisingly Simple Self-Supervision for Offline Reinforcement Learningarxiv.org
• Self-CTRL: Self-Consistency Training with Reinforcement Learningarxiv.org
• [2606.05555] Representation Learning Enables Scalable Multitask Deep Reinforcement Learningarxiv.org
• Robotic World Model: A Neural Network Simulator for Robust Policy Optimization in Roboticsarxiv.org
• Emergent Dexterity via Diverse Resets and Large-Scale Reinforcement Learningarxiv.org
• ERCEIVER-ACTOR: A Multi-Task Transformer for Robotic Manipulationarxiv.org
• Neural MP: A Generalist Neural Motion Plannerarxiv.org
• Deep Reinforcement Learning from Human Preferencesproceedings.neurips.cc
• [2506.22401] Exploration from a Primal-Dual Lens: Value-Incentivized Actor-Critic Methods for Sample-Efficient Online RLarxiv.org
• Learning to Walk in Minutes Using Massively Parallel Deep Reinforcement Learningarxiv.org
• Contrastive Learning as Goal-Conditioned Reinforcement Learningarxiv.org