Controlling blood glucose variability under uncertainty using reinforcement learning and Gaussian processes

Abstract

Automated control of blood glucose (BG) concentration with a fully automated artificial pancreas will certainly improve the quality of life for insulin-dependent patients. Closed-loop insulin delivery is challenging due to inter- and intra-patient variability, errors in glucose sensors and delays in insulin absorption. Responding to the varying activity levels seen in outpatients, with unpredictable and unreported food intake, and providing the necessary personalized control for individuals is a challenging task for existing control algorithms. A novel approach for controlling glycemic variability using simulation-based learning is presented. A policy iteration algorithm that combines reinforcement learning with Gaussian process approximation is proposed. To account for multiple sources of uncertainty, a control policy is learned off-line using an Ito's stochastic model of the glucose-insulin dynamics. For safety and performance, only relevant data are sampled through Bayesian active learning. Results obtained demonstrate that a generic policy is both safe and efficient for controlling subject-specific variability due to a patient's lifestyle and its distinctive metabolic response.