Improving Rates of Annual Transcranial Doppler Screening in Children with Sickle Cell Disease: A Quality Improvement Project

Abstract

Background: Transcranial doppler (TCD) screening identifies individuals with sickle cell disease (SCD) at an increased risk for stroke between the ages of 2 and 16. Providers at Boston Children's Hospital recognized an opportunity to improve adherence to the TCD screening guidelines. The Specific, Measurable, Achievable, Relevant, and Time-Bound (SMART) aim of our quality improvement (QI) initiative was to increase the proportion of eligible patients receiving a TCD within 15 months of their last TCD to >95% by July 1st 2022. Methods: This QI project was conducted at an academic pediatric hospital. We used the Model for Improvement as the improvement framework. To understand the current state, a TCD data repository was created in July 2020. We identified patients with SCD eligible for a TCD via our institution's data warehouse. Patients who received their primary hematology care at an alternative institution, and those who received chronic transfusions or definitive curative therapy for SCD were excluded. TCD adherence was defined as receiving a TCD within three months of the TCD being due (12-15 months from the last TCD). Our database was refined with a multidisciplinary group to determine the baseline TCD adherence rate of interest. Multiple plan-do-study-act (PDSA) cycles were performed to identify individuals overdue for screening and test interventions to increase adherence. An electronic notification was sent to providers on overdue patients, initially quarterly, then monthly. This evolved to sending proactive notifications 3 months ahead of when patients would be overdue for a TCD. Monthly adherence rates were tracked, with the data from May 2021 through June 2022 measured via a run chart (Figure 1a). In addition, we used logistic regression to identify variables potentially associated with non-adherence to the recommended TCD screening. Using TCD eligible patients from July 1, 2021 to June 30, 2022 a simple logistic regression model was fit with the following variables: age, gender, poverty, childhood opportunity index (COI) score, distance to the hospital, need for an interpreter, insurance type, and SCD genotype. All tests were two tailed and p<0.05 was considered significant. Using the same variables as input we developed numerous machine learning (ML) models to predict TCD adherence. The top three performing models were chosen for further optimization and were compared using the algorithm's accuracy, area under the receive operator curve, F1 statistic, precision and recall. Confusion matrices were used to summarize model output. Feature importance for the top performing model was determined using Xgboost. All analyses were performed using RStudio version 1.4.1717, Google Colab, and Stata. Results: Our baseline TCD adherence rate was 71%. The average TCD screening rate from June 1, 2021 to June 30, 2022 was 94% (Figure 1a). Using the cohort of patients followed from June 1, 2021-June 30 2022 we identified 127 patients eligible for TCD; 48 patients were overdue for their TCD, a 37.8% "ever-missed TCD screening” rate. In univariate analysis, age, gender, neighborhood poverty level, COI score, distance to the hospital, need for an interpreter, insurance type, and SCD genotype were not predictive of likelihood to be overdue for TCD screening. Using multivariate analysis, there were no statistically significant relationships between sociodemographic variables and likelihood of missed TCD. The need for an interpreter and distance to the hospital trended towards significance with odds ratios of 0.261 (p=0.068) and 1.016 (p=0.06) respectively (Figure 1b). The top performing ML model was a random forest classifier. This model had the following performance metrics: accuracy=0.88, ROC AUC=0.70, F1=0.73, precision=0.80, and recall=0.67. The features that were most predictive of TCD adherence were age, distance from the hospital and COI score. Conclusion: Already high TCD screening rates can be improved with a dedicated QI initiative. Our next intervention will prospectively evaluate our ML algorithm to predict patients at risk of being overdue for their TCD and help us achieve our goal of consistently > 95% TCD adherence. We will provide targeted outreach to these patients to mitigate patient level barriers to obtaining a TCD such as focused education directed towards older patients and their families and transportation vouchers/parking passes for TCD associated visits. Figure 1View largeDownload PPTFigure 1View largeDownload PPT Close modal