Long-term use of the hybrid artificial pancreas by adjusting carbohydrate ratios and programmed basal rate: A reinforcement learning approach

Abstract

Background and objectivesThe hybrid artificial pancreas regulates glucose levels in people with type 1 diabetes. It delivers (i) insulin boluses at meal times based on the meals’ carbohydrate content and the carbohydrate ratios (CRs) and (ii) insulin basal, between meals and at night, continuously modulated around individual-specific programmed basal rate. The CRs and programmed basal rate significantly vary between individuals and within the same individual with type 1 diabetes, and using suboptimal values in the hybrid artificial pancreas may degrade glucose control. We propose a reinforcement learning algorithm to adaptively optimize CRs and programmed basal rate to improve the performance of the hybrid artificial pancreas. MethodsThe proposed reinforcement learning algorithm was designed using the Q-learning approach. The algorithm learns the optimal actions (CRs and programmed basal rate) by applying them to the individual's state (previous day's glucose levels and insulin delivery) based on an exploration and exploitation trade-off. First, outcomes from our simulator were compared to those of a clinical study in 23 individuals with type 1 diabetes and have yielded similar results. Second, the learning algorithm was tested using the simulator with two scenarios. Scenario 1 has fixed meal sizes and ingestion times and scenario 2 has a more realistic eating behavior with random meal sizes, ingestion times, and carbohydrate counting errors. ResultsAfter about five weeks, the reinforcement learning algorithm improved the percentage of time spent in target range from 67% to 86.7% in scenario 1 and 65.5% to 86% in scenario 2. The percentage of time spent below 4.0 mmol/L decreased from 9% to 0.9% in scenario 1 and 9.5% to 1.1% in scenario 2. ConclusionsResults indicate that the proposed algorithm has the potential to improve glucose control in people with type 1 diabetes using the hybrid artificial pancreas. The proposed algorithm is a key in making the hybrid artificial pancreas adaptive for the long-term real life outpatient studies.