Abstract
BackgroundAccurate, pragmatic risk stratification for postoperative delirium (POD) is necessary to target preventative resources toward high-risk patients. Machine learning (ML) offers a novel approach to leveraging electronic health record (EHR) data for POD prediction. We sought to develop and internally validate a ML-derived POD risk prediction model using preoperative risk features, and to compare its performance to models developed with traditional logistic regression.MethodsThis was a retrospective analysis of preoperative EHR data from 24,885 adults undergoing a procedure requiring anesthesia care, recovering in the main post-anesthesia care unit, and staying in the hospital at least overnight between December 2016 and December 2019 at either of two hospitals in a tertiary care health system. One hundred fifteen preoperative risk features including demographics, comorbidities, nursing assessments, surgery type, and other preoperative EHR data were used to predict postoperative delirium (POD), defined as any instance of Nursing Delirium Screening Scale ≥2 or positive Confusion Assessment Method for the Intensive Care Unit within the first 7 postoperative days. Two ML models (Neural Network and XGBoost), two traditional logistic regression models (“clinician-guided” and “ML hybrid”), and a previously described delirium risk stratification tool (AWOL-S) were evaluated using the area under the receiver operating characteristic curve (AUC-ROC), sensitivity, specificity, positive likelihood ratio, and positive predictive value. Model calibration was assessed with a calibration curve. Patients with no POD assessments charted or at least 20% of input variables missing were excluded.ResultsPOD incidence was 5.3%. The AUC-ROC for Neural Net was 0.841 [95% CI 0. 816–0.863] and for XGBoost was 0.851 [95% CI 0.827–0.874], which was significantly better than the clinician-guided (AUC-ROC 0.763 [0.734–0.793], p < 0.001) and ML hybrid (AUC-ROC 0.824 [0.800–0.849], p < 0.001) regression models and AWOL-S (AUC-ROC 0.762 [95% CI 0.713–0.812], p < 0.001). Neural Net, XGBoost, and ML hybrid models demonstrated excellent calibration, while calibration of the clinician-guided and AWOL-S models was moderate; they tended to overestimate delirium risk in those already at highest risk.ConclusionUsing pragmatically collected EHR data, two ML models predicted POD in a broad perioperative population with high discrimination. Optimal application of the models would provide automated, real-time delirium risk stratification to improve perioperative management of surgical patients at risk for POD.
Highlights
Postoperative delirium (POD) is a common and serious complication of surgery [1], and is associated with numerous adverse events including prolonged length of stay, more frequent institutional discharge, higher readmission rates, functional decline, dependency in activities of daily living, and cognitive decline [2,3,4,5,6,7,8,9]
Whereas existing delirium prediction models tend to rely on well-known delirium risk factors such as age and cognitive impairment [23,24,25], Machine learning (ML) allows for analysis of patterns in large amounts of data pragmatically collected in the electronic health record (EHR) to identify higher-order interactions that would be difficult to identify through traditional data analysis techniques [26]
The area under the receiver operating characteristic curve (AUC-Receiver Operating Characteristic (ROC)) was 0.840 and 0.841 by DeLong’s method (DL) for Neural Network (Table 2, Fig. 2A.)
Summary
Postoperative delirium (POD) is a common and serious complication of surgery [1], and is associated with numerous adverse events including prolonged length of stay, more frequent institutional discharge, higher readmission rates, functional decline, dependency in activities of daily living, and cognitive decline [2,3,4,5,6,7,8,9]. Machine learning-derived risk prediction models have been developed to predict delirium in hospitalized patients [17], postoperative delirium in focused patient populations [18], non-delirium-related intraoperative complications [19, 20], and postoperative mortality [21], in addition to applications in many other contexts [22]. Increasingly feasible realtime EHR-based applications of ML-derived predictions in clinical practice [22, 26] have the potential to conserve valuable human resources through automation of risk stratification procedures, since use of existing risk stratification tools have often required too much clinician input to enter clinical workflow [24]. We sought to develop and internally validate a ML-derived POD risk prediction model using preoperative risk features, and to compare its performance to models developed with traditional logistic regression
Sign-in/Register to view highlights & summary Talk to us
Join us for a 30 min session where you can share your feedback and ask us any queries you have
Schedule a callDisclaimer: All third-party content on this website/platform is and will remain the property of their respective owners and is provided on "as is" basis without any warranties, express or implied. Use of third-party content does not indicate any affiliation, sponsorship with or endorsement by them. Any references to third-party content is to identify the corresponding services and shall be considered fair use under The CopyrightLaw.
