Quantifying misclassification and bias errors due to hierarchical sepsis scores in real-time sepsis diagnosis

Abstract

Severe sepsis and septic shock are common causes of intensive care unit (ICU) admission, length of stay, mortality and cost. Sepsis is very difficult to diagnose in real-time and early treatment can show positive benefits. Current sepsis scores are hierarchically calculated, creating significant case-control and/or misclassification biases in developing real-time diagnostic classifiers based on real-time clinical data. This study examines the impact on kernel density classifier performance of a continuous sepsis score derived from a hierarchical score and based on current well-accepted metrics.Clinical data and model-based insulin sensitivity from 36 patients in the Christchurch Hospital ICU (6071 h) is used to segregate severe sepsis and septic shock from mild sepsis and SIRS using a modified continuous sepsis score using a kernel density estimator (KDE) method. Best case resubstitution, worst case bootstrap, and expected case .632 bootstrap estimates are calculated. Diagnostic quality is assessed using likelihood ratios (LHRs), ROC curves, diagnostic odds-ratios (DOR), sensitivity and specificity. Results were compared to those using the hierarchical sepsis score and the same KDE.The continuous sepsis score yielded smoother, more realistic time-varying trajectories for sepsis evolution. The classifier provided very good results, potentially able to change clinical decisions, but did not reach the best diagnostic accuracy for all performance metrics. Using several metrics avoided assessment bias, and likelihood ratios provided the best overall assessment relative to clinical decision making processes. Finally, quantifying infection presence is a source of case-control and misclassification bias in classifier development, showing clear need for improved sepsis scores.