External validation of two clinical prediction models for mortality in COVID-19 patients (4C and NEWS2), in three centers in Medellín, Colombia: Assessing the impact of vaccination over time

Abstract

ObjectivesExternal validation of the 4C and NEWS2 scores for the prediction of in-hospital mortality in COVID-19 patients, and evaluation of its operational performance in two time periods: before and after the start of the vaccination program in Colombia. MethodsRetrospective cohort in three high complexity hospitals in the city of Medellín, Colombia, between June 2020 and April 2022. ResultsThe areas under the ROC curve (AUC) for the 4C mortality risk score and the NEWS2 were 0.75 (95% CI 0.73–0.78) and 0.68 (95% CI 0.66–0.71), respectively. For the 4C score, the AUC for the first and second periods was 0.77 (95% CI 0.74–0.80) and 0.75 (95% CI 0.71–0.78); whilst for the NEWS2 score, it was 0.68 (95% CI 0.65–0.71) and 0.69 (95% CI 0.64–0.73). The calibration for both scores was adequate, albeit with reduced performance during the second period. ConclusionsThe 4C mortality risk score proved to be the more adequate predictor of in-hospital mortality in COVID-19 patients in this Latin American population. The operational performance during both time periods remained similar, which shows its utility notwithstanding major changes, including vaccination, as the pandemic evolved.