Critical care outcome prediction equation model, version 7

This study aimed to construct machine learning (ML) models for predicting prolonged length of stay (pLOS) in intensive care units (ICU) among general ICU patients. A multicenter database called eICU (Collaborative Research Database) was used for model derivation and internal validation, and the Medical Information Mart for Intensive Care (MIMIC) III database was used for external validation. We used four different ML methods (random forest, support vector machine, deep learning, and gradient boosting decision tree (GBDT)) to develop prediction models. The prediction performance of the four models were compared with the customized simplified acute physiology score (SAPS) II. The area under the receiver operation characteristic curve (AUROC), area under the precision-recall curve (AUPRC), estimated calibration index (ECI), and Brier score were used to measure performance. In internal validation, the GBDT model achieved the best overall performance (Brier score, 0.164), discrimination (AUROC, 0.742; AUPRC, 0.537), and calibration (ECI, 8.224). In external validation, the GBDT model also achieved the best overall performance (Brier score, 0.166), discrimination (AUROC, 0.747; AUPRC, 0.536), and calibration (ECI, 8.294). External validation showed that the calibration curve of the GBDT model was an optimal fit, and four ML models outperformed the customized SAPS II model. The GBDT-based pLOS-ICU prediction model had the best prediction performance among the five models on both internal and external datasets. Furthermore, it has the potential to assist ICU physicians to identify patients with pLOS-ICU risk and provide appropriate clinical interventions to improve patient outcomes.

This study examined records of 2566 consecutive COVID-19 patients at five Massachusetts hospitals and sought to predict level-of-care requirements based on clinical and laboratory data. Several classification methods were applied and compared against standard pneumonia severity scores. The need for hospitalization, ICU care, and mechanical ventilation were predicted with a validation accuracy of 88%, 87%, and 86%, respectively. Pneumonia severity scores achieve respective accuracies of 73% and 74% for ICU care and ventilation. When predictions are limited to patients with more complex disease, the accuracy of the ICU and ventilation prediction models achieved accuracy of 83% and 82%, respectively. Vital signs, age, BMI, dyspnea, and comorbidities were the most important predictors of hospitalization. Opacities on chest imaging, age, admission vital signs and symptoms, male gender, admission laboratory results, and diabetes were the most important risk factors for ICU admission and mechanical ventilation. The factors identified collectively form a signature of the novel COVID-19 disease.

Decision letter: Early prediction of level-of-care requirements in patients with COVID-19

Are Unselected Risk Scores in the Cardiac Intensive Care Unit Needed?

This study reviewed the concepts and properties of the receiver operating characteristic (ROC) curve and precision recall (PR) curve, and made suggestions on the application of two curves based on the prevalence in combination with the results of simulation data. This study demonstrated that the ROC curve and PR curve had different properties, which could reflect the performance of diagnostic methods from various aspects. These two curves should be selected with a consideration of prevalence and clinical scenarios. When the prevalence was less than 20%, especially less than 5%, the PR curve could be adopted.

To develop a mortality prediction model for critically ill patients based on multidimensional and dynamic clinical data collected by the hospital information system (HIS) using random forest algorithm, and to compare the prediction efficiency of the model with acute physiology and chronic health evaluation II (APACHE II) model. The clinical data of 10 925 critically ill patients aged over 14 years old admitted to the Third Xiangya Hospital of Central South University from January 2014 to June 2020 were extracted from the HIS system, and APACHE II scores of the critically ill patients were extracted. Expected mortality of patients was calculated according to the death risk calculation formula of APACHE II scoring system. A total of 689 samples with APACHE II score records were used as the test set, and the other 10 236 samples were used to establish the random forest model, of which 10% (n = 1 024) were randomly selected as the validation set and 90% (n = 9 212) were selected as the training set. According to the time series of 3 days before the end of critical illness, the clinical characteristics of patients such as general information, vital signs data, biochemical test results and intravenous drug doses were selected to develope a random forest model for predicting the mortality of critically ill patients. Using the APACHE II model as a reference, receiver operator characteristic curve (ROC curve) was drawn, and the discrimination performance of the model was evaluated through the area under the ROC curve (AUROC). According to the precision and recall, Precision-Recall curve (PR curve) was drawn, and the calibration performance of the model was evaluated through the area under the PR curve (AUPRC). Calibration curve was drawn, and the consistency between the predicted event occurrence probability of the model and the actual occurrence probability was evaluated through the calibration index Brier score. Among the 10 925 patients, there were 7 797 males (71.4%) and 3 128 females (28.6%). The average age was (58.9±16.3) years old. The median length of hospital stay was 12 (7, 20) days. Most patients (n = 8 538, 78.2%) were admitted to intensive care unit (ICU), and the median length of ICU stay was 66 (13, 151) hours. The hospitalized mortality was 19.0% (2 077/10 925). Compared with the survival group (n = 8 848), the patients in the death group (n = 2 077) were older (years old: 60.1±16.5 vs. 58.5±16.4, P < 0.01), the ratio of ICU admission was higher [82.8% (1 719/2 077) vs. 77.1% (6 819/8 848), P < 0.01], and the proportion of patients with hypertension, diabetes and stroke history was also higher [44.7% (928/2 077) vs. 36.3% (3 212/8 848), 20.0% (415/2 077) vs. 16.9% (1 495/8 848), 15.5% (322/2 077) vs. 10.0% (885/8 848), all P < 0.01]. In the test set data, the prediction value of random forest model for the risk of death during hospitalization of critically ill patients was greater than that of APACHE II model, which showed by that the AUROC and AUPRC of random forest model were higher than those of APACHE II model [AUROC: 0.856 (95% confidence interval was 0.812-0.896) vs. 0.783 (95% confidence interval was 0.737-0.826), AUPRC: 0.650 (95% confidence interval was 0.604-0.762) vs. 0.524 (95% confidence interval was 0.439-0.609)], and Brier score was lower than that of APACHE II model [0.104 (95% confidence interval was 0.085-0.113) vs. 0.124 (95% confidence interval was 0.107-0.141)]. The random forest model based on multidimensional dynamic characteristics has great application value in predicting hospital mortality risk for critically ill patients, and it is superior to the traditional APACHE II scoring system.

Delirium poses significant risks to patients, but countermeasures can be taken to mitigate negative outcomes. Accurately forecasting delirium in intensive care unit (ICU) patients could guide proactive intervention. Our primary objective was to predict ICU delirium by applying machine learning to clinical and physiologic data routinely collected in electronic health records. Two prediction models were trained and tested using a multicenter database (years of data collection 2014 to 2015), and externally validated on two single-center databases (2001 to 2012 and 2008 to 2019). The primary outcome variable was delirium defined as a positive Confusion Assessment Method for the ICU screen, or an Intensive Care Delirium Screening Checklist of 4 or greater. The first model, named "24-hour model," used data from the 24 h after ICU admission to predict delirium any time afterward. The second model designated "dynamic model," predicted the onset of delirium up to 12 h in advance. Model performance was compared with a widely cited reference model. For the 24-h model, delirium was identified in 2,536 of 18,305 (13.9%), 768 of 5,299 (14.5%), and 5,955 of 36,194 (11.9%) of patient stays, respectively, in the development sample and two validation samples. For the 12-h lead time dynamic model, delirium was identified in 3,791 of 22,234 (17.0%), 994 of 6,166 (16.1%), and 5,955 of 28,440 (20.9%) patient stays, respectively. Mean area under the receiver operating characteristics curve (AUC) (95% CI) for the first 24-h model was 0.785 (0.769 to 0.801), significantly higher than the modified reference model with AUC of 0.730 (0.704 to 0.757). The dynamic model had a mean AUC of 0.845 (0.831 to 0.859) when predicting delirium 12 h in advance. Calibration was similar in both models (mean Brier Score [95% CI] 0.102 [0.097 to 0.108] and 0.111 [0.106 to 0.116]). Model discrimination and calibration were maintained when tested on the validation datasets. Machine learning models trained with routinely collected electronic health record data accurately predict ICU delirium, supporting dynamic time-sensitive forecasting.

Forecasting acute kidney injury and resource utilization in ICU patients using longitudinal, multimodal models

Background:Advances in artificial intelligence (AI) have realized the potential of revolutionizing healthcare, such as predicting disease progression via longitudinal inspection of Electronic Health Records (EHRs) and lab tests from patients admitted to Intensive Care Units (ICU). Although substantial literature exists addressing broad subjects, including the prediction of mortality, length-of-stay, and readmission, studies focusing on forecasting Acute Kidney Injury (AKI), specifically dialysis anticipation like Continuous Renal Replacement Therapy (CRRT) are scarce. The technicality of how to implement AI remains elusive.Objective:This study aims to elucidate the important factors and methods that are required to develop effective predictive models of AKI and CRRT for patients admitted to ICU, using EHRs in the Medical Information Mart for Intensive Care (MIMIC) database.Methods:We conducted a comprehensive comparative analysis of established predictive models, considering both time-series measurements and clinical notes from MIMIC-IV databases. Subsequently, we proposed a novel multi-modal model which integrates embeddings of top-performing unimodal models, including Long Short-Term Memory (LSTM) and BioMedBERT, and leverages both unstructured clinical notes and structured time series measurements derived from EHRs to enable the early prediction of AKI and CRRT.Results:Our multimodal model achieved a lead time of at least 12 hours ahead of clinical manifestation, with an Area Under the Receiver Operating Characteristic Curve (AUROC) of 0.888 for AKI and 0.997 for CRRT, as well as an Area Under the Precision Recall Curve (AUPRC) of 0.727 for AKI and 0.840 for CRRT, respectively, which significantly outperformed the baseline models. Additionally, we performed a SHapley Additive exPlanation (SHAP) analysis using the expected gradients algorithm, which highlighted important, previously underappreciated predictive features for AKI and CRRT.Conclusion:Our study revealed the importance and the technicality of applying longitudinal, multimodal modeling to improve early prediction of AKI and CRRT, offering insights for timely interventions. The performance and interpretability of our model indicate its potential for further assessment towards clinical applications, to ultimately optimize AKI management and enhance patient outcomes.

Nosocomial pneumonia (NP) and ventilator associated pneumonia (VAP) have been associated with financially significant economic burden and increased case fatality rate in adult intensive care units (ICUs). This study was designed to evaluate case fatality rate among patients with NP and VAP in a respiratory ICU. In 2008-2013, VAP and NP in the ICUs were included in this retrospective single-centre cohort study. Data on demographics, co-morbidities, severity of illness, mechanical ventilation, empirical treatment, length of hospital stay and laboratory findings were recorded in each group, as were case fatality rate during ICU admission and after discharge including short-term (28-day) and long-term (a year) case fatality rate. A total of 108 patients with VAP (n = 64, median (IQR) age: 70 (61-75) years, 67.2% were men) or NP (n = 44, median (IQR) age: 68 (62-74) years, 68.2% were men) were found. Appropriate empirical antibiotic therapy was identified only in 45.2 and 42.9% of patients with VAP and NP, respectively. Overall case fatality rate in VAP and NP (81.3 vs 84.1), ICU case fatality rate (42.2 vs 45.5%), short-term case fatality rate (15.6 vs 27.3%) and long-term case fatality rate (23.4 vs 11.4%) were similar between VAP and NP groups along with occurrence 50% of case fatality rate cases in the first 2 months and 90% within the first year of discharge. Multivariate analysis showed that chronic obstructive pulmonary disease (COPD) (HR: 3.15, 95% CI: 1.06-9.38; p = 0.039) and presence of septic shock (HR: 3.83, 95% CI: 1.26-11.60; p = 0.018) were independently associated with lower survival. In conclusion, our findings in a retrospective cohort of respiratory ICU patients with VAP or NP revealed high ICU, short- and long-term case fatality rates within 1 year of diagnosis, regardless of the diagnosis of NP after 48 h of initial admission or after induction of ventilator support. COPD and presence of septic shock are associated with high fatality rate and our findings speculate that as increasing compliance with infection control programs and close monitoring especially in 2 months of discharge might reduce high-case fatality rate in patients with VAP and NP.

Accurate classification of COVID-19 patients with different severity via machine learning.

In many two-class problems in automated classification and information retrieval, the classes are imbalanced, and the separation between the positive and negative classes is large. The precision-recall (PR) curve has been suggested as an alternative to the receiver operating characteristic (ROC) curve to characterize the performance of automated systems when the classes are imbalanced, and the area under the precision-recall curve (AUCPR) has been suggested as an alternative performance measure to the area under the ROC curve (AUCROC). AUCPR and AUCROC are distinct measures of performance, even though the relationship between the precision-recall and ROC curves is well-known. In this study, we compared the statistical power of the AUCPR to that of the AUCROC. Our results indicate that the AUCPR can offer a small statistical advantage when the prevalence is low and the separation between the positive and negative classes is large. When the data set is more balanced or the separation between the classes is low or moderate, AUCROC has slightly higher power.

PurposeWhilst survival in paediatric critical care has improved, clinicians lack tools capable of predicting long-term outcomes. We developed a machine learning model to predict poor school outcomes in children surviving intensive care unit (ICU).MethodsPopulation-based study of children < 16 years requiring ICU admission in Queensland, Australia, between 1997 and 2019. Failure to meet the National Minimum Standard (NMS) in the National Assessment Program-Literacy and Numeracy (NAPLAN) assessment during primary and secondary school was the primary outcome. Routine ICU information was used to train machine learning classifiers. Models were trained, validated and tested using stratified nested cross-validation.Results13,957 childhood ICU survivors with 37,200 corresponding NAPLAN tests after a median follow-up duration of 6 years were included. 14.7%, 17%, 15.6% and 16.6% failed to meet NMS in school grades 3, 5, 7 and 9. The model demonstrated an Area Under the Receiver Operating Characteristic curve (AUROC) of 0.8 (standard deviation SD, 0.01), with 51% specificity to reach 85% sensitivity [relative Area Under the Precision Recall Curve (rel-AUPRC) 3.42, SD 0.06]. Socio-economic status, illness severity, and neurological, congenital, and genetic disorders contributed most to the predictions. In children with no comorbidities admitted between 2009 and 2019, the model achieved a AUROC of 0.77 (SD 0.03) and a rel-AUPRC of 3.31 (SD 0.42).ConclusionsA machine learning model using data available at time of ICU discharge predicted failure to meet minimum educational requirements at school age. Implementation of this prediction tool could assist in prioritizing patients for follow-up and targeting of rehabilitative measures.

Background: Emergence of current pandemic caused by novel SARS–COV–2 has already caused over 963000 deaths. Case fatality rate (CFR) estimation helps understanding the disease severity and the lethality trend, high risk population and subsequently, optimization of quality healthcare facilities. Our observational study aimed to find out existing trends in treating the most vulnerable group with scarce medical resource allocation and to implement necessary support services to comply with the ensuing need for best possible outcomes in our ICU. Methodology: In this observational study, all COVID–19 diagnosed patients admitted in our ICU from July 4, 2020 to September 22, 2020, were enrolled. Data were obtained from the core ICU register of Bangabandhu Sheikh Mujib Medical University (BSMMU), Dhaka, Bangladesh. Information accumulated on predesigned data sheets comprised of particulars of patients, co–morbidities, duration of ICU stay, mode of oxygenation, organ support and quick SOFA scores. Total deaths in ICU (in hospital or referred from outside of BSMMU) were recorded. Results: The results revealed that all patients were either very severe or critically sick with COVID–19 pneumonia at the time of ICU admission. Out of 174 patients, 46 (26.44%) were put on invasive ventilation and the rest received noninvasive ventilation in the form of NRM, high flow nasal cannula (HFNC), continuous positive airway pressure (CPAP or BiPAP), CTEX CPAP and non–invasive ventilation (NIV) as appropriate. Male and female ratio was 74:26. Age of patients ranged between 19–95y. The median age of patients was 65 y (IQR: 57–70).Quick SOFA scores were more than 2 in 65.37% of patients. Regarding co–existing organ dysfunction 13.8% had 3 or more co–morbidities; while 74.1% had 2 and 9.8% had a single systemic illness along with COVID–19. Most common diseases encountered among 135 deceased were hypertension (64%), IHD (49%), diabetes mellitus (45%), bronchial asthma or COPD (32%), renal failure (either ARF or CRF) (20%). Overall CFR due to COVID–19 pneumonia associated with co–morbidities was 77.6%. Relatively higher CFR (82.6%) was evident harboring multi–organ dysfunction especially among COVID–19 patients aged 50y or more. Gender linked CFR were 81.4% and 66.7% in males and females respectively. Conclusion: High CFR demonstrates significant correlation with increasing age and co–morbidities and survival functions. Late presentation to the hospital and invasive mechanical ventilation also contributed to high CFR. Keywords: Case fatality rate; CFR; COVID–19; Intensive Care Unit; Survival function Abbreviations: CFR – Case fatality rate; NRM – Non-rebreathing mask; HFNC – high flow nasal cannula; CPAP – continuous positive airway pressure; BiPAP – Bi-level positive airway pressure; NIV – Non–invasive ventilation; IHD – Ischemic heart disease; ARF – Acute renal failure; CRF – Chronic renal failure Citation: Islam MS, Bhowmick DK, Parveen M, Kamal MM, Akhtaruzzaman AKM. Case fatality rate and survival functions of severe COVID–19 patients in intensive care unit of Bangabandhu Sheikh Mujib Medical University in Bangladesh: an observational study. Anaesth. pain intensive care 2021;25(4):443–449. DOI: Received: June 4, 2021, Reviewed: June 7, 2021, Accepted: June 12, 2021

CORR Synthesis: When Should We Be Skeptical of Clinical Prediction Models?