Development and validation of a model for surveillance of postoperative bleeding complications using structured electronic health records data

Abstract

BackgroundPostoperative bleeding complications surveillance is done primarily through manual chart review. The purpose of this study was to develop and validate a detection model for postoperative bleeding complications using structured electronic health records data. MethodsPatients who underwent operations at 1 of 5 hospitals within our local health system between 2013 and 2019 and whose complications were reported by the American College of Surgeons National Surgical Quality Improvement Program were included. Electronic health records data were linked to American College of Surgeons National Surgical Quality Improvement Program data using personal health identifiers. Electronic health records predictors included diagnosis codes mapped to PheCodes, procedure names, and medications within 30 days after surgery. We defined bleeding events as the transfusion of red blood cell components within 30 days after surgery. The knockoff filter and the lasso were used to develop a model in a training set of operations from January 2013 to March 2017. Performance of each model was tested in a held-out data set of patients who underwent operations from March 2017 to October 2019. ResultsA total of 30,639 patients were included; 1,112 patients (3.6%) had a bleeding event. Eight predictor variables were selected by the knockoff filter. When applied to the test set, specificity was 94%, sensitivity was 94%, area under the curve was 0.97, and accuracy was 93%. Calibration was consistent in lower predicted risk patients, whereas the model slightly overpredicted risk in high-risk patients. ConclusionWe created a parsimonious, accurate model for identifying patients with bleeding complications. This model can be used to augment manual chart review for surveillance and reporting of perioperative bleeding complications, enabling inclusion of all surgeries in quality improvement efforts.