Why digital health trials can fail: Lessons learned from a randomized trial of health coaching and virtual cardiac rehabilitation

Abstract

BackgroundWe performed a trial to evaluate the efficacy of a blended intervention with personalized health coaching and virtual cardiac rehabilitation to improve medication adherence and risk factors. The trial was terminated early. Here, we describe findings from a root cause analysis and lessons learned.MethodsSmartGUIDE was an open-label, single-center trial that randomized participants with coronary artery disease who were prescribed a statin and/or P2Y12 inhibitor 1:1 to either usual care or the added use of a mobile app with components of cardiac rehabilitation paired with personal virtual coaching. The primary outcome was medication adherence: proportion of days covered (PDC). The planned sample size was 132. We performed a root cause analysis to evaluate processes from study development to closure.ResultsDuring trial conduct, the technology start-up withdrew the intervention. The study was terminated early with 63 participants randomized and data from 26 available for analysis. The median PDC was high in both groups (intervention group 94%, interquartile range [IQR] 88%–96%; control group: 99%, IQR 95%–100%). Root cause analysis identified factors for not achieving trial objectives: key factors that limited enrollment (inclusion criteria, low penetration of compatible smartphones), participant retention or engagement (poor app product, insufficient technology support), and suboptimal choice of a technology partner (technology start-up’s inexperience in health care, poor product design, inadequate fundraising).ConclusionWe identified important and preventable factors leading to trial failure. These factors may be common across digital health trials and may explain prior observations that many such trials are never completed. Careful vetting of technology partners and more pragmatic study designs may prevent these missteps.