Abstract
BackgroundSeverity scores assess the acuity of critical illness by penalizing for the deviation of physiologic measurements from normal and aggregating these penalties (also called “weights” or “subscores”) into a final score (or probability) for quantifying the severity of critical illness (or the likelihood of in-hospital mortality). Although these simple additive models are human readable and interpretable, their predictive performance needs to be further improved.MethodsWe present OASIS +, a variant of the Oxford Acute Severity of Illness Score (OASIS) in which an ensemble of 200 decision trees is used to predict in-hospital mortality based on the 10 same clinical variables in OASIS.ResultsUsing a test set of 9566 admissions extracted from the MIMIC-III database, we show that OASIS + outperforms nine previously developed severity scoring methods (including OASIS) in predicting in-hospital mortality. Furthermore, our results show that the supervised learning algorithms considered in our experiments demonstrated higher predictive performance when trained using the observed clinical variables as opposed to OASIS subscores.ConclusionsOur results suggest that there is room for improving the prognostic accuracy of the OASIS severity scores by replacing the simple linear additive scoring function with more sophisticated non-linear machine learning models such as RF and XGB.
Highlights
Severity scores assess the acuity of critical illness by penalizing for the deviation of physiologic meas‐ urements from normal and aggregating these penalties into a final score for quantifying the severity of critical illness
Using a test set of intensive care unit (ICU) stays lasting at least for 24 h that represent 9566 distinct adult patients, we show that Oxford Acute Severity of Illness Score (OASIS) + outperforms nine previously developed severity scoring methods in predicting in-hospital mortality
Our major contributions are as follows: (1) We introduce an improved variant of the OASIS severity score with substantial improvements in predictive and prognostic performance; (2) We assess the performance of nine severity scores on predicting in-hospital mortality and demonstrate superior performance of OASIS + over all of them; (3) We show that OASIS thresholds for transforming clinical measurements into subscores are not optimal for the MIMIC-III data; (4) We release the implementation of the OASIS + model as an online tool for predicting in-hospital mortality as well as the associated Python scripts for benchmarking OASIS + using independent test data from other health systems
Summary
Severity scores assess the acuity of critical illness by penalizing for the deviation of physiologic meas‐ urements from normal and aggregating these penalties ( called “weights” or “subscores”) into a final score (or prob‐ ability) for quantifying the severity of critical illness (or the likelihood of in-hospital mortality) These simple additive models are human readable and interpretable, their predictive performance needs to be further improved. Dysfunction Score (LODS) [11], systemic inflammatory response syndrome (SIRS) [12], Sequential Organ Failure Assessment (SOFA) [13], and Oxford Acute Severity of Illness Score (OASIS) [14] These severity scoring methods have been widely used in many research and clinical applications including predicting mortality, length of stay (LoS), stratifying patients for clinical trials, and evaluation of ICU quality of care [15]. The expected improvement in performance might introduce a tradeoff in model interpretability resulting from the increased model complexity which might hamper clinicians’ ability to associate explanations with the predictions made by the model
Sign-in/Register to view highlights & summary Talk to us
Join us for a 30 min session where you can share your feedback and ask us any queries you have
Schedule a callDisclaimer: All third-party content on this website/platform is and will remain the property of their respective owners and is provided on "as is" basis without any warranties, express or implied. Use of third-party content does not indicate any affiliation, sponsorship with or endorsement by them. Any references to third-party content is to identify the corresponding services and shall be considered fair use under The CopyrightLaw.
