49-LB: Reported Utility of Automated Blood Glucose Forecasts

Abstract

Background: A growing number of people use apps to track diabetes data and identify retrospective patterns that can inform management behavior. In September 2018, One Drop launched Automated Decision Support, which predicts blood glucose 8 hours in the future for non-insulin-dependent users with type 2 diabetes (T2D). Predictions help inform diabetes decision-making prospectively, but only if users find them useful. Objective: To understand how predictive insights facilitate behavior change in people with T2D, One Drop collected feedback from users and trained a machine learning model to predict when forecasts would be perceived as useful. This model was used to select timing and content of future forecasts and accompanying support messages. We then measured changes in feedback to assess forecast utility. Methodology: An initial sample of 23,876 forecasts were sent via in-app notifications to 4679 users with T2D. Forecasts consisted of BG trend, duration, and level (“rising, but not too high, over the next 3 hours”) and, when appropriate, a support message relevant to forecast and user history. Forecast delivery was random, triggered with 50% probability when information was logged, no more than once/day/user. Forecasts could be rated “Useful” or “Not Useful.” A machine learning model, trained on the initial sample, predicted the probability of each type of feedback. A second sample of 28838 forecasts were sent to 5506 users, with probability of usefulness determined by the model. Results: In the 1st sample, 42.8% of forecasts received feedback from 69.6% of users; 87.1% was “Useful.” In the 2nd sample, 63.7% of forecasts received feedback from 67.1% of users; 92.4% was “Useful.” Discussion: A new machine learning model tailored forecast delivery, reducing the “Not Useful” rate by 41.1% (from 12.9% of feedback to 7.6%). Correlations between useful forecasts and behavior change will be explored in future work. Disclosure D.R. Goldner: Employee; Self; One Drop. Stock/Shareholder; Self; One Drop. M. Heyman: Consultant; Self; Lexicon Pharmaceuticals, Inc., Lilly Diabetes, Tandem Diabetes Care. Employee; Self; One Drop. A. Hirsch: Employee; Self; Informed Data Systems Inc. Research Support; Self; Fitbit, Inc., MannKind Corporation. Stock/Shareholder; Self; Informed Data Systems Inc. C.Y. Osborn: Employee; Self; One Drop. O. Lubling: Employee; Self; One Drop. B. Huddleston: Employee; Self; One Drop. R.L. Vlaha: Employee; Self; One Drop. K. Prenger: Employee; Self; One Drop. R. Murthy: Employee; Self; One Drop. J. Starke: Employee; Self; One Drop. J. Briceno: Employee; Self; One Drop. D. Ledvina: Employee; Self; One Drop. J. Dachis: Stock/Shareholder; Self; Informed Data Systems Inc.