Generalizability of predictive models for Clostridioides difficile infection, severity and recurrence at an urban safety-net hospital

Abstract

IntroductionPredictive models for Clostridioides difficile (C. difficile) infection can identify high risk patients and aid clinicians in preventing infection. Issues of generalisability regarding current predictive models have been acknowledged but to our knowledge have never been quantified. MethodsThrough case-control sampling from an urban safety-net hospital, we created C. difficile infection, severity, and recurrence predictive models using multivariate logistic regression. Models were validated using 5-fold cross-validation and inverse probability weights (IPW) based on two different catchment area definitions were used to improve external validity. Akaike Information Criterion (AIC), area under the receiver operating characteristic curve (AUROC), sensitivity, and specificity with bootstrapped confidence intervals were used to assess and compare model fit and performance. ResultsChanges in performance before and after weighting were small across all models; although, differences were more apparent after weighting the recurrence model (AUROCs of 0.78, 0.76, 0.71 for the unweighted and two weighted models, respectively). Overall, the infection model performed the best (AUROC=0.82 [95%CI: 0.78, 0.85]) followed by the recurrence model (AUROC=0.78 [95%CI: 0.69, 0.86]) and then the severity model (AUROC=0.70 [95%CI: 0.63, 0.78]. ConclusionsThe performance of our models after weighting did not change drastically suggesting that our model predicting C. difficile infection, severity, and recurrence may not be impacted by patient selection factors; however, other researchers may want to consider addressing these catchment forces using IPW.