Logistic regression technique is comparable to complex machine learning algorithms in predicting cognitive impairment related to post intensive care syndrome

Abstract

To evaluate the performance of machine learning (ML) models and to compare it with logistic regression (LR) technique in predicting cognitive impairment related to post intensive care syndrome (PICS-CI). We conducted a prospective observational study of ICU patients at two tertiary hospitals. A cohort of 2079 patients was screened, and finally 481 patients were included. Seven different ML models were considered, decision tree (DT), random forest (RF), XGBoost, neural network (NN), naïve bayes (NB), and support vector machine (SVM), and compared with logistic regression (LR). Discriminative ability was evaluated by area under the receiver operating characteristic curve (AUC), calibration belt plots, and Hosmer–Lemeshow test was used to assess calibration. Decision curve analysis was performed to quantify clinical utility. Duration of delirium, poor Richards–Campbell sleep questionnaire (RCSQ) score, advanced age, and sepsis were the most frequent and important candidates risk factors for PICS-CI. All ML models showed good performance (AUC range: 0.822–0.906). NN model had the highest AUC (0.906 [95% CI 0.857–0.955]), which was slightly higher than, but not significantly different from that of LR (0.898 [95% CI 0.847–0.949]) (P > 0.05, Delong test). Given the overfitting and complexity of some ML models, the LR model was then used to develop a web-based risk calculator to aid decision-making (https://model871010.shinyapps.io/dynnomapp/). In a low dimensional data, LR may yield as good performance as other complex ML models to predict cognitive impairment after ICU hospitalization.