This article presents a significant advancement in the field of deep brain stimulation (DBS) through the development of a closed-loop system that integrates reinforcement learning (RL) and neural simulation techniques. Our study specifically addresses the treatment of Parkinson’s disease (PD), a chronic neurodegenerative disorder characterized by debilitating motor and non-motor symptoms. We introduce a novel basal ganglia-thalamic (BGT) model, designed to serve as an interactive environment for RL applications, facilitating the optimization of stimulation parameters in real-time based on feedback from the patient’s neural activity.
Four distinct RL frameworks—Soft Actor-Critic (SAC), Twin Delayed Deep Deterministic Policy Gradient (TD3), Proximal Policy Optimization (PPO), and Advantage Actor-Critic (A2C)—were meticulously implemented and evaluated. Among these, the TD3 agent demonstrated a remarkable 67% reduction in average power dissipation compared to conventional open-loop DBS systems, while effectively preserving the physiological responses of the BGT circuitry. This reduction in power consumption is critical for enhancing the longevity and safety of DBS devices, which are often limited by battery life.
The findings underscore the potential of RL to provide adaptive, patient-specific adjustments to stimulation parameters, thereby improving therapeutic outcomes and minimizing adverse effects associated with overstimulation. By leveraging RL techniques, the proposed system aims to enhance energy efficiency and therapeutic efficacy, ultimately paving the way for more personalized and effective treatments in clinical settings. This research not only contributes to the optimization of cl-DBS algorithms but also lays the groundwork for future clinical applications, promising to enhance the quality of life for individuals suffering from movement disorders.
Authors: Chia-Hung Cho, Pin-Jui Huang, Meng-Chao Chen, Chii-Wann Lin