Advanced decision support system for individuals with diabetes on multiple daily injections therapy using reinforcement learning and nearest-neighbors: In-silico and clinical results

Abstract

Many individuals with diabetes on multiple daily insulin injections therapy use carbohydrate ratios (CRs) and correction factors (CFs) to determine mealtime and correction insulin boluses. The CRs and CFs vary over time due to physiological changes in individuals' response to insulin. Errors in insulin dosing can lead to life-threatening abnormal glucose levels, increasing the risk of retinopathy, neuropathy, and nephropathy. Here, we present a novel learning algorithm that uses Q-learning to track optimal CRs and uses nearest-neighbors based Q-learning to track optimal CFs. The learning algorithm was compared with the run-to-run algorithm A and the run-to-run algorithm B, both proposed in the literature, over an 8-week period using a validated simulator with a realistic scenario created with suboptimal CRs and CFs values, carbohydrate counting errors, and random meals sizes at random ingestion times. From Week 1 to Week 8, the learning algorithm increased the percentage of time spent in target glucose range (4.0 to 10.0 mmol/L) from 51 % to 64 % compared to 61 % and 58 % with the run-to-run algorithm A and the run-to-run algorithm B, respectively. The learning algorithm decreased the percentage of time spent below 4.0 mmol/L from 9 % to 1.9 % compared to 3.4 % and 2.3 % with the run-to-run algorithm A and the run-to-run algorithm B, respectively. The algorithm was also assessed by comparing its recommendations with (i) the endocrinologist's recommendations on two type 1 diabetes individuals over a 16-week period and (ii) real-world individuals' therapy settings changes of 23 individuals (19 type 2 and 4 type 1) over an 8-week period using the commercial Bigfoot Unity Diabetes Management System. The full agreements (i) were 89 % and 76 % for CRs and CFs for the type 1 diabetes individuals and (ii) was 62 % for mealtime doses for the individuals on the commercial Bigfoot system. Therefore, the proposed algorithm has the potential to improve glucose control in individuals with type 1 and type 2 diabetes.