Automation of Hematopoietic Cell Transplantation Outcomes Reporting Leads to Dramatic Reduction in Errors Reported to Real-World Data Registry

Abstract

Institutions that perform hematopoietic cell transplantation (HCT) are required by law to report standardized, structured data on transplantation outcomes. A key post-transplantation outcome is engraftment, the time between HCT infusion and reemergence of circulating neutrophils and platelets. At our center, we found that manual chart abstraction for engraftment data was highly error-prone. We developed a custom R/Shiny application that automatically calculates engraftment dates and displays them in an intuitive format to augment the manual chart review. Our hypothesis was that use of the application to assist with calculating and reporting engraftment dates would be associated with a decreased error rate. The study was conducted at a single tertiary care institution. The application was developed in a collaborative, multidisciplinary fashion by members of an embedded cellular therapy informatics team. Retrospective validation of the application's accuracy was conducted on all malignant HCTs from February 2016 to December 2020 (n=198). Real-world use of the application was evaluated prospectively from April 2021 through April 2022 (n=53). The Welch 2-sample t test was used to compare error rates preimplementation and postimplementation. Data were visualized using p charts, and standard special cause variation rules were applied. The accuracy of reported data postdeployment increased dramatically; the engraftment error rate decreased from 15% to 3.8% for neutrophils (P=.003) and from 28% to 1.9% for platelets (P < .001). This study demonstrates the effective deployment of a custom R/Shiny application that was associated with significantly reduced error rates in HCT engraftment reporting for operational, research, and regulatory purposes. Users reported subjective satisfaction with the application and that it addressed difficulties with the legacy manual process. Identifying and correcting erroneous data in engraftment reporting could lead to a more efficient and accurate nationwide assessment of transplantation success. Furthermore, we show that it is possible and practical for academic medical centers to create and support embedded informatics teams that can quickly build applications for clinical operations in a manner compliant with regulatory requirements.