ARLPE: A meta reinforcement learning framework for glucose regulation in type 1 diabetics

Abstract

External artificial pancreas with autonomous control algorithms has proved its effectiveness in glucose regulation for type 1 diabetes. Nonetheless, most existing algorithms cannot adapt to unknown patients with limited clinical data. To achieve the automatic glucose regulation of unknown patients even with uncertainties and noises, we propose Active Reinforcement Learning with Personalized Embeddings (ARLPE) for normoglycemia maintenance. Our framework contains a meta-training period and a fine-tuning period. The meta-training period aims to learn: (1) a generalized policy for glucose regulation and (2) a probabilistic encoder that summarizes the personalized information and context into an embedding. The fine-tuning period is designed to generate a personalized policy for the unknown patient with the help of an active learning module to explore valuable experiences. Experiments on multiple patients demonstrate that our algorithm can not only converge blood glucose to the normoglycemic bounds and avoid hypoglycemia but also achieve the glucose regulation of a new unacquainted patient using limited BG data (only 25 samples). ARLPE achieves the time in range (TIR) score of 98.63% and 97.93% in adult and adolescent cohorts, respectively, significantly outperforming the state-of-the-art competing methods for glucose regulation. It shows the great potential of generating personalized clinical strategies for diabetics.