Automated Calculator for the Pediatric Sequential Organ Failure Assessment Score: Development and External Validation in a Single-Center 7-Year Cohort, 2015-2021.

Abstract

The pediatric Sequential Organ Failure Assessment (pSOFA) score summarizes severity of organ dysfunction and can be used to predict in-hospital mortality. Manual calculation of the pSOFA score is time-consuming and prone to human error. An automated method that is open-source, flexible, and scalable for calculating the pSOFA score directly from electronic health record data is desirable. Single-center, retrospective cohort study. Quaternary 40-bed PICU. All patients admitted to the PICU between 2015 and 2021 with ICU stay of at least 24 hours. None. We used 77 records to evaluate the automated score. The automated algorithm had an overall accuracy of 97%. The algorithm calculated the respiratory component of two cases incorrectly. An expert human annotator had an initial accuracy of 75% at the patient level and 95% at the component level. An untrained human annotator with general clinical research experience had an overall accuracy of 16% and component-wise accuracy of 67%. Weighted kappa for agreement between the automated method and the expert annotator's initial score was 0.92 (95% CI, 0.88-0.95), and between the untrained human annotator and the automated score was 0.50 (95% CI, 0.36-0.61). Data from 9146 patients (in-hospital mortality 3.6%) were included to validate externally the discriminability of the automated pSOFA score. The admission-day pSOFA score had an area under the receiver operating characteristic curve of 0.79 (95% CI, 0.77-0.82). The developed automated algorithm calculates pSOFA score with high accuracy and is more accurate than a trained expert rater and nontrained data abstracter. pSOFA's performance for predicting in-hospital mortality was lower in our cohort than it was for the originally derived score.