A New Glycemic closed-loop control based on Dyna-Q for Type-1-Diabetes

Abstract

Objective:Type 1 Diabetes Mellitus is an autoimmune disease which requires constant care from patients. Continuous Glucose Monitoring (CGM) devices allow to keep track of the glycemic trend for 24 h a day. Control algorithms are necessary to automate the therapy and to develop an artificial pancreas system. The objective of this study is to develop a fully-automated glycemic control system based on a Dyna-Q Reinforcement Learning algorithm that is able to automatically decide the insulin infusion without the need of carbohydrate information from the patient. Methods:A Dyna-Q Reinforcement Learning architecture is proposed to automate glycemic control while using only information on past CGM and insulin data, validated on data from 10 in silico patients. Results:The proposed glycemic predictor achieves an average RMSE and a MARD of 13.2 mg/dl and 6.9% on 10 virtual adults, and of 15.0 mg/dl and 7.2% on 12 real patients, while over 98.8% of the forecasts fall within the safe zones of the Clarke Error Grid. The controller is able to maintain the glucose levels of the virtual subjects in the target range for 60.7% of the simulation time on a 24 h scenario, without causing hypoglycemic events in 8 out of 10 patients. Conclusion:The proposed architecture is able to achieve good performance without exploiting information on carbohydrates and using a much smaller amount of training data compared to models in the literature. Significance:We proved that model-based Reinforcement Learning could be a valid approach for a human-safe fully-automated artificial pancreas.