Brier Score Research Articles

Anthracycline-induced cardiovascular toxicity: validation of the Heart Failure Association and International Cardio-Oncology Society risk score.

Baseline cardiovascular toxicity risk stratification is critical in cardio-oncology. The Heart Failure Association (HFA) and International Cardio-Oncology Society (ICOS) score aims to assess this risk but lacks real-life validation. This study validates the HFA-ICOS score for anthracycline-induced cardiovascular toxicity. Anthracycline-treated patients in the CARDIOTOX registry (NCT02039622) were stratified by the HFA-ICOS score. The primary endpoint was symptomatic or moderate to severe asymptomatic cancer therapy-related cardiac dysfunction (CTRCD), with all-cause mortality and cardiovascular mortality as secondary endpoints. The analysis included 1066 patients (mean age 54 ± 14 years; 81.9% women; 24.5% ≥65 years). According to the HFA-ICOS criteria, 571 patients (53.6%) were classified as low risk, 333 (31.2%) as moderate risk, 152 (14.3%) as high risk, and 10 (0.9%) as very high risk. Median follow-up was 54.8 months (interquartile range 24.6-81.8). A total of 197 patients (18.4%) died, and 718 (67.3%) developed CTRCD (symptomatic: n = 45; moderate to severe asymptomatic: n = 24; and mild asymptomatic: n = 649). Incidence rates of symptomatic or moderate to severe symptomatic CTRCD and all-cause mortality significantly increased with HFA-ICOS score [hazard ratio 28.74, 95% confidence interval (CI) 9.33-88.5; P < .001, and hazard ratio 7.43, 95% CI 3.21-17.2; P < .001) for very high-risk patients. The predictive model demonstrated good calibration (Brier score 0.04, 95% CI 0.03-0.05) and discrimination (area under the curve 0.78, 95% CI 0.70-0.82; Uno's C-statistic 0.78, 95% CI 0.71-0.84) for predicting symptomatic or severe/moderate asymptomatic CTRCD at 12 months. The HFA-ICOS score effectively categorizes patients by cardiovascular toxicity risk and demonstrates strong predictive ability for high-risk anthracycline-related cardiovascular toxicity and all-cause mortality.

Development and validation of novel interpretable survival prediction models based on drug exposures for severe heart failure during vulnerable period

BackgroundSevere heart failure (HF) has a higher mortality during vulnerable period while targeted predictive tools, especially based on drug exposures, to accurately assess its prognoses remain largely unexplored. Therefore, this study aimed to utilize drug information as the main predictor to develop and validate survival models for severe HF patients during this period.MethodsWe extracted severe HF patients from the MIMIC-IV database (as training and internal validation cohorts) as well as from the MIMIC-III database and local hospital (as external validation cohorts). Three algorithms, including Cox proportional hazards model (CoxPH), random survival forest (RSF), and deep learning survival prediction (DeepSurv), were applied to incorporate the parameters (partial hospitalization information and exposure durations of drugs) for constructing survival prediction models. The model performance was assessed mainly using area under the receiver operator characteristic curve (AUC), brier score (BS), and decision curve analysis (DCA). The model interpretability was determined by the permutation importance and Shapley additive explanations values.ResultsA total of 11,590 patients were included in this study. Among the 3 models, the CoxPH model ultimately included 10 variables, while RSF and DeepSurv models incorporated 24 variables, respectively. All of the 3 models achieved respectable performance metrics while the DeepSurv model exhibited the highest AUC values and relatively lower BS among these models. The DCA also verified that the DeepSurv model had the best clinical practicality.ConclusionsThe survival prediction tools established in this study can be applied to severe HF patients during vulnerable period by mainly inputting drug treatment duration, thus contributing to optimal clinical decisions prospectively.

A two-stage ensemble learning based prediction and grading model for PD-1/PD-L1 inhibitor-related cardiac adverse events: A multicenter retrospective study

BackgroundImmune-related cardiac adverse events (ircAEs) caused by programmed cell death protein-1 (PD-1) and programmed death-ligand-1 (PD-L1) inhibitors can lead to fulminant and even fatal consequences. This study aims to develop a prediction and grading model for ircAEs, enabling graded management of patients. MethodsThis study utilized medical record systems from two medical institutions to develop a prediction and grading model for ircAEs using ten machine learning algorithms and two variable screening methods. The model was developed based on a two-stage ensemble learning framework. In the first stage, the ircAEs and non-ircAEs cases were classified. In the second stage, ircAEs cases were grouped into grades 1–2 and 3–5. The experiments were evaluated using five-fold cross-validation. The model's prediction performance was assessed using accuracy, precision, recall, F1 value, Brier score, receiver operating characteristic curve area (AUC), and area under the precision-recall curve (AUPR). Results615 patients were included in the study. 147 experienced ircAEs, and 44 experienced grade 3–5 ircAEs. The soft voting classifier trained using the variables screened by feature importance ranking performed better than other classifiers in both stages. The average AUC for the first and second stages is 84.18 % and 85.13 %, respectively. In the first stage, the three most important variables are N-terminal B-type natriuretic peptide (NT-proBNP), interleukin-2 (IL-2), and C-reactive protein (CRP). In the second stage, the patient's age, NT-proBNP, and left ventricular ejection fraction (LVEF) are the three most critical variables. ConclusionsThe prediction and grading model of ircAEs based on two-stage ensemble learning established in this study has good performance and potential clinical application.

Modeling of the cascading impacts of drought and forest fire based on a Bayesian network

The cascading impact of disastrous events has become an important focus of disaster research. In recent years, forest fires have occurred frequently in China, causing huge economic losses. Studying the cascading impacts of drought and forest fire is of great significance for reducing disaster risks. Taking Yunnan Province of China as an example, meteorological data and forest fire point data from 2005 to 2018 were collected and statistically analyzed. A Bayesian network model of the cascading impacts of drought and forest fire was established, enabling the prior probability and conditional probability of nodes to be determined. Based on this information, a probability prediction was established using causal reasoning. Finally, in the case test, the Brier score was used to test the accuracy of the model. The Brier test value was 0.305, which was less than the qualified threshold of 0.6. The results indicated that the Bayesian network model established in this study had a good prediction performance, which was basically consistent with the facts. The results provide an insight into the mechanism by which drought induced forest fires occur and will be of use in forest fire prevention work.

A Dynamic Prognostic Model for Identifying Vulnerable COVID-19 Patients at High Risk of Rapid Deterioration.

We aimed to validate and, if performance was unsatisfactory, update the previously published prognostic model to predict clinical deterioration in patients hospitalized for COVID-19, using data following vaccine availability. Using electronic health records of patients ≥18 years, with laboratory-confirmed COVID-19, from a large care-delivery network in Massachusetts, USA, from March 2020 to November 2021, we tested the performance of the previously developed prediction model and updated the prediction model by incorporating data after availability of COVID-19 vaccines. We randomly divided data into development (70%) and validation (30%) cohorts. We built a model predicting worsening in a published severity scale in 24 h by LASSO regression and evaluated performance by c-statistic and Brier score. Our study cohort consisted of 8185 patients (Development: 5730 patients [mean age: 62; 44% female] and Validation: 2455 patients [mean age: 62; 45% female]). The previously published model had suboptimal performance using data after November 2020 (N = 4973, c-statistic = 0.60. Brier score = 0.11). After retraining with the new data, the updated model included 38 predictors including 18 changing biomarkers. Patients hospitalized after Jun 1st, 2021 (when COVID-19 vaccines became widely available in Massachusetts) were younger and had fewer comorbidities than those hospitalized before. The c-statistic and Brier score were 0.77 and 0.13 in the development cohort, and 0.73 and 0.14 in the validation cohort. The characteristics of patients hospitalized for COVID-19 differed substantially over time. We developed a new dynamic model for rapid progression with satisfactory performance in the validation set.

Justice-Involved Status and In-Hospital Mortality Among Nonelderly Adults During the COVID-19 Pandemic, 2021

ObjectiveThe purpose of this research is to examine the role of justice-involved status on in-hospital mortality among nonelderly adults during the second year of the COVID-19 pandemic. MethodsThis retrospective cohort study used data from the 2021 State Inpatient Databases for 20 US states, which included discharges from general acute care hospitals among adults aged 18-64 years hospitalized for at least 24 hours. The main outcome was all-cause in-hospital mortality and the primary comparison was justice-involved status. We used logistic regression to estimate the odds ratios and 95% confidence intervals (CIs), with adjustment for sociodemographic factors, Elixhauser comorbidities, COVID-19 diagnosis, admission acuity, other clinical features, metropolitan area, and seasonality. We randomly split the data into a 50% training and 50% validation set. With the latter, we evaluated the performance of our final model. ResultsThe study population included 4,712,441 discharges (1.1% justice-involved; mean [SD] age 47.5 [12.8] years; 47.0% women; 63.6% White, 21.8% Black, 11.8% Hispanic, 1.8% Asian/Pacific Islander, and 1.0% American Indian/Alaska Native). Among these, 102,735 in-hospital deaths (2.2%) occurred. In the multivariate analysis, in-hospital mortality was about 40% less likely among justice-involved patients (odds ratios 0.6, 95% CI 0.5-0.7, P value <0.01). The final validated model showed excellent discrimination (area under the curve for the receiver operator characteristic 0.953, 95% CI 0.952-0.954) and good calibration (Brier score 0.014, calibration belt P value .186). ConclusionsIn this cohort study, justice-involved status was independently associated with lower in-hospital mortality. Future studies should examine preadmission and postdischarge outcomes.

Application of the Unbalanced Ensemble Algorithm for Prognostic Prediction Outcomes of All-Cause Mortality in Coronary Heart Disease Patients Comorbid with Hypertension.

This study sought to develop an unbalanced-ensemble model that could accurately predict death outcomes of patients with comorbid coronary heart disease (CHD) and hypertension and evaluate the factors contributing to death. Medical records of 1058 patients with coronary heart disease combined with hypertension and excluding those acute coronary syndrome were collected. Patients were followed-up at the first, third, sixth, and twelfth months after discharge to record death events. Follow-up ended two years after discharge. Patients were divided into survival and nonsurvival groups. According to medical records, gender, smoking, drinking, COPD, cerebral stroke, diabetes, hyperhomocysteinemia, heart failure and renal insufficiency of the two groups were sorted and compared and other influencing factors of the two groups, feature selection was carried out to construct models. Owing to data unbalance, we developed four unbalanced-ensemble prediction models based on Balanced Random Forest (BRF), EasyEnsemble, RUSBoost, SMOTEBoost and the two base classification algorithms based on AdaBoost and Logistic. Each model was optimised using hyperparameters based on GridSearchCV and evaluated using area under the curve (AUC), sensitivity, recall, Brier score, and geometric mean (G-mean). Additionally, to understand the influence of variables on model performance, we constructed a SHapley Additive explanation (SHAP) model based on the optimal model. There were significant differences in age, heart rate, COPD, cerebral stroke, heart failure and renal insufficiency in the nonsurvival group compared with the survival group. Among all models, BRF yielded the highest AUC (0.810; 95% CI, 0.778-0.839), sensitivity (0.990; 95% CI, 0.981-1.000), recall (0.990; 95% CI, 0.981-1.000), and G-mean (0.806; 95% CI, 0.778-0.827), and the lowest Brier score (0.181; 95% CI, 0.178-0.185). Therefore, we identified BRF as the optimal model. Furthermore, red blood cell count (RBC), body mass index (BMI), and lactate dehydrogenase were found to be important mortality-associated risk factors. BRF combined with advanced machine learning methods and SHAP is highly effective and accurately predicts mortality in patients with CHD comorbid with hypertension. This model has the potential to assist clinicians in modifying treatment strategies to improve patient outcomes.

Cureit: An End-to-End Pipeline for Implementing Mixture Cure Models With an Application to Liposarcoma Data.

Cure models are a useful alternative to Cox proportional hazards models in oncology studies when there is a subpopulation of patients who will not experience the event of interest. Although software is available to fit cure models, there are limited tools to evaluate, report, and visualize model results. This article introduces the cureit R package, an end-to-end pipeline for building mixture cure models, and demonstrates its use in a data set of patients with primary extremity and truncal liposarcoma. To assess associations between liposarcoma histologic subtypes and disease-specific death (DSD) in patients treated at Memorial Sloan Kettering Cancer Center between July 1982 and September 2017, mixture cure models were fit and evaluated using the cureit package. Liposarcoma histologic subtypes were defined as well-differentiated, dedifferentiated, myxoid, round cell, and pleomorphic. All other analyzed liposarcoma histologic subtypes were significantly associated with higher DSD in cure models compared with well-differentiated. In multivariable models, myxoid (odds ratio [OR], 6.25 [95% CI, 1.32 to 29.6]) and round cell (OR, 16.2 [95% CI, 2.80 to 93.2]) liposarcoma had higher incidences of DSD compared with well-differentiated patients. By contrast, dedifferentiated liposarcoma was associated with the latency of DSD (hazard ratio, 10.6 [95% CI, 1.48 to 75.9]). Pleomorphic liposarcomas had significantly higher risk in both incidence and the latency of DSD (P < .0001). Brier scores indicated comparable predictive accuracy between cure and Cox models. We developed the cureit pipeline to fit and evaluate mixture cure models and demonstrated its clinical utility in the liposarcoma disease setting, shedding insights on the subtype-specific associations with incidence and/or latency.

Ensemble Algorithm for Risk Prediction of Clinical Failure After Anterior Cruciate Ligament Reconstruction.

Patient-specific risk profiles of clinical failure after anterior cruciate ligament reconstruction (ACLR) are meaningful for preoperative surgical planning and postoperative rehabilitation guidance. To create an ensemble algorithm machine learning (ML) model and ML-based web-based tool that can predict the patient-specific risk of clinical failure after ACLR. Cohort study; Level of evidence, 3. Included were 432 patients (mean age, 26.8 ± 8.4 years; 74.1% male) who underwent anatomic double-bundle ACLR with hamstring tendon autograft between January 2010 and February 2019. The primary outcome was the probability of clinical failure at a minimum 2-year follow-up. The authors included 24 independent variables for feature selection and model development. The data set was split randomly into training sets (75%) and test sets (25%). Models were built using 4 ML algorithms: extreme gradient boosting, random forest, light gradient boosting machine, and adaptive boosting. In addition, a weighted-average voting (WAV) ensemble model was constructed using the ensemble-voting technique to predict clinical failure after ACLR. Concordance (area under the receiver operating characteristic curve [AUC]), calibration, and decision curve analysis were used to evaluate predictive performances of the 5 models. Clinical failure occurred in 73 of the 432 patients (16.9%). The 8 most important predictors for clinical failure were follow-up period, high-grade preoperative knee laxity, time from injury to ACLR, participation in competitive sports, posterior tibial slope, graft diameter, age at surgery, and medial meniscus resection. The WAV ensemble algorithm achieved the best predictive performance based on concordance (AUC, 0.9139), calibration (calibration intercept, -0.1806; calibration slope, 1.2794; Brier score, 0.0888), and decision curve analysis (greatest net benefits) and was used to develop an web-based application to predict a patient's clinical failure risk of ACLR. The WAV ensemble algorithm was able to accurately predict patient-specific risk of clinical failure after ACLR. Clinicians and patients can use the web-based application during preoperative consultation to understand individual prediction outcomes.

Advancing precision medicine in immunoglobulin light-chain amyloidosis: a novel prognostic model incorporating multi-organ indicators.

To develop a more accurate prognostic model that incorporates indicators of multi-organ involvement for immunoglobulin light-chain (AL) Amyloidosis patients. Biopsy-proven AL amyloidosis patients between January 1, 2012, and February 28, 2023, were enrolled and randomly divided into a training set and a test set at a ratio of 7:3. Prognostic indicators that comprehensively cover cardiac, renal, and hepatic involvement were identified in the training set by random survival forest (RSF). Then, RSF and Cox models were established. The Concordance index (C-index) and integrated brier scores (IBS) were applied to evaluate the models' performance in the test set. Besides, the net reclassification index (NRI) and integrated discrimination improvement (IDI) were calculated. A total of 173 eligible patients were included. After a median follow-up of 25.9 (9.2, 50.3) months, 48 (27.7%) patients died. Creatine kinase-MB, estimated glomerular filtration rate ≤ 50mL/min/1.73m2, interventricular septum ≥ 15mm, ejection fraction, alanine aminotransferase and Live involved were selected to develop prediction models. The RSF model based on the above indicators achieved C-index and IBS values of 0.834 (95% CI 0.725-0.915) and 0.151 (95% CI 0.1402-0.181), respectively. At last, the NRI and IDI of the RSF model were 0.301 (95% CI 0.048-0.546, P = 0.012) and 0.157 (95% CI 0.041-0.269, P < 0.001) at 5-year by comparing the RSF model with the Cox model which is based on the Mayo 2012 staging system. The RSF model that incorporates indicators of multi-organ involvement had a great performance, which may be helpful for physicians' decision-making and more accurate overall survival prediction.

Development and validation of a nomogram model for predicting venous thromboembolism risk in lung cancer patients treated with immune checkpoint inhibitors: A cohort study in China.

Venous thromboembolism (VTE) poses a significant threat to lung cancer patients, particularly those receiving treatment with immune checkpoint inhibitors (ICIs). We aimed to develop and validate a nomogram model for predicting the occurrence of VTE in lung cancer patients undergoing ICI therapy. The data for this retrospective cohort study was collected from cancer patients admitted to Chongqing University Cancer Hospital for ICI treatment between 2019 and 2022. The research data is divided into training and validation sets using a 7:3 ratio. Univariate and multivariate analyses were employed to identify risk factors for VTE. Based on these analyses, along with clinical expertise, a nomogram model was crafted. The model's predictive accuracy was assessed through receiver operating characteristic (ROC) curves, calibration curves, decision curve analysis, clinical impact curve, and other relevant metrics. The initial univariate analysis pinpointed 13 potential risk factors for VTE. The subsequent stepwise multivariate regression analysis identified age, Karnofsky performance status, chemotherapy, targeted, platelet count, lactate dehydrogenase, monoamine oxidase, D-dimer, fibrinogen, and white blood cell count as significant predictors of VTE. These 10 variables were the foundation for a predictive model, illustrated by a clear and intuitive nomogram. The model's discriminative ability was demonstrated by the ROC curve, which showed an area under the curve of 0.815 (95% CI 0.772-0.858) for the training set, and 0.753 (95% CI 0.672-0.835) for the validation set. The model's accuracy was further supported by Brier scores of 0.068 and 0.080 for the training and validation sets, respectively, indicating a strong correlation with actual outcomes. We have successfully established and validated a nomogram model for predicting VTE risk in lung cancer patients treated with ICIs.

SHAP based predictive modeling for 1 year all-cause readmission risk in elderly heart failure patients: feature selection and model interpretation

Heart failure (HF) is a significant global public health concern with a high readmission rate, posing a serious threat to the health of the elderly population. While several studies have used machine learning (ML) to develop all-cause readmission risk prediction models for elderly patients with HF, few have integrated ML-selected features with those chosen by human experts to assess HF patients readmission. A retrospective analysis of 8396 elderly HF patients hospitalized at the Affiliated Hospital of North Sichuan Medical College from January 1, 2018 to December 31, 2021 was conducted. Variables selected by XGBoost, LASSO regression, and random forest constituted the machine group, while the human expert group comprised variables chosen by two experienced cardiovascular professors. The variables selected by both groups were combined to form a human–machine collaboration group. Model performance was evaluated using the area under the receiver operating characteristic curve (AUC). The SHapley Additive exPlanations (SHAP) method was used to elucidate the importance of each predictive feature, explain the impact of individual features on the model, and provide visual representation. A total of 73 features were included for model development. The human–machine collaboration model, utilizing CatBoost, achieved an AUC of 0.83617, an F1-score of 0.73521, and a Brier score of 0.16536 on the validation set. This model demonstrated superior predictive performance compared to those created solely by human experts or machine. The SHAP plot was then used to visually display the feature analysis of the human–machine collaboration model, revealing HGB, NT-proBNP, smoking history, NYHA classification, and LVEF as the 5 most important features. This study indicate that the human–machine collaboration model outperforms those relying solely on human expert selection or machine algorithm at predicting all-cause readmission in elderly HF patients. The application of the SHAP method enhanced the interpretability of the model outcomes, aiding clinicians in accurately pinpointing risk factors associated with HF readmission. This advancement enables the formulation of tailored treatment strategies, offering a more personalized approach to patient care.

Interpretable machine learning models for predicting clinical pregnancies associated with surgical sperm retrieval from testes of different etiologies: a retrospective study

BackgroundThe relationship between surgical sperm retrieval of different etiologies and clinical pregnancy is unclear. We aimed to develop a robust and interpretable machine learning (ML) model for predicting clinical pregnancy using the SHapley Additive exPlanation (SHAP) association of surgical sperm retrieval from testes of different etiologies.MethodsA total of 345 infertile couples who underwent intracytoplasmic sperm injection (ICSI) treatment with surgical sperm retrieval due to different etiologies from February 2020 to March 2023 at the reproductive center were retrospectively analyzed. The six machine learning (ML) models were used to predict the clinical pregnancy of ICSI. After evaluating the performance characteristics of the six ML models, the Extreme Gradient Boosting model (XGBoost) was selected as the best model, and SHAP was utilized to interpret the XGBoost model for predicting clinical pregnancies and to reveal the decision-making process of the model.ResultsCombining the area under the receiver operating characteristic curve (AUROC), accuracy, precision, recall, F1 score, brier score, and the area under the precision-recall (P-R) curve (AP), the XGBoost model has the best performance (AUROC: 0.858, 95% confidence interval (CI): 0.778–0.936, accuracy: 79.71%, brier score: 0.151). The global summary plot of SHAP values shows that the female age is the most important feature influencing the model output. The SHAP plot showed that younger age in females, bigger testicular volume (TV), non-tobacco use, higher anti-müllerian hormone (AMH), lower follicle-stimulating hormone (FSH) in females, lower FSH in males, the temporary ejaculatory disorders (TED) group, and not the non-obstructive azoospermia (NOA) group all resulted in an increased probability of clinical pregnancy.ConclusionsThe XGBoost model predicts clinical pregnancies associated with testicular sperm retrieval of different etiologies with high accuracy, reliability, and robustness. It can provide clinical counseling decisions for patients with surgical sperm retrieval of various etiologies.

Machine-learning-based prediction of disability progression in multiple sclerosis: An observational, international, multi-center study.

Disability progression is a key milestone in the disease evolution of people with multiple sclerosis (PwMS). Prediction models of the probability of disability progression have not yet reached the level of trust needed to be adopted in the clinic. A common benchmark to assess model development in multiple sclerosis is also currently lacking. Data of adult PwMS with a follow-up of at least three years from 146 MS centers, spread over 40 countries and collected by the MSBase consortium was used. With basic inclusion criteria for quality requirements, it represents a total of 15, 240 PwMS. External validation was performed and repeated five times to assess the significance of the results. Transparent Reporting for Individual Prognosis Or Diagnosis (TRIPOD) guidelines were followed. Confirmed disability progression after two years was predicted, with a confirmation window of six months. Only routinely collected variables were used such as the expanded disability status scale, treatment, relapse information, and MS course. To learn the probability of disability progression, state-of-the-art machine learning models were investigated. The discrimination performance of the models is evaluated with the area under the receiver operator curve (ROC-AUC) and under the precision recall curve (AUC-PR), and their calibration via the Brier score and the expected calibration error. All our preprocessing and model code are available at https://gitlab.com/edebrouwer/ms_benchmark, making this task an ideal benchmark for predicting disability progression in MS. Machine learning models achieved a ROC-AUC of 0⋅71 ± 0⋅01, an AUC-PR of 0⋅26 ± 0⋅02, a Brier score of 0⋅1 ± 0⋅01 and an expected calibration error of 0⋅07 ± 0⋅04. The history of disability progression was identified as being more predictive for future disability progression than the treatment or relapses history. Good discrimination and calibration performance on an external validation set is achieved, using only routinely collected variables. This suggests machine-learning models can reliably inform clinicians about the future occurrence of progression and are mature for a clinical impact study.

Machine learning-enabled prediction of prolonged length of stay in hospital after surgery for tuberculosis spondylitis patients with unbalanced data: a novel approach using explainable artificial intelligence (XAI)

BackgroundTuberculosis spondylitis (TS), commonly known as Pott’s disease, is a severe type of skeletal tuberculosis that typically requires surgical treatment. However, this treatment option has led to an increase in healthcare costs due to prolonged hospital stays (PLOS). Therefore, identifying risk factors associated with extended PLOS is necessary. In this research, we intended to develop an interpretable machine learning model that could predict extended PLOS, which can provide valuable insights for treatments and a web-based application was implemented.MethodsWe obtained patient data from the spine surgery department at our hospital. Extended postoperative length of stay (PLOS) refers to a hospitalization duration equal to or exceeding the 75th percentile following spine surgery. To identify relevant variables, we employed several approaches, such as the least absolute shrinkage and selection operator (LASSO), recursive feature elimination (RFE) based on support vector machine classification (SVC), correlation analysis, and permutation importance value. Several models using implemented and some of them are ensembled using soft voting techniques. Models were constructed using grid search with nested cross-validation. The performance of each algorithm was assessed through various metrics, including the AUC value (area under the curve of receiver operating characteristics) and the Brier Score. Model interpretation involved utilizing methods such as Shapley additive explanations (SHAP), the Gini Impurity Index, permutation importance, and local interpretable model-agnostic explanations (LIME). Furthermore, to facilitate the practical application of the model, a web-based interface was developed and deployed.ResultsThe study included a cohort of 580 patients and 11 features include (CRP, transfusions, infusion volume, blood loss, X-ray bone bridge, X-ray osteophyte, CT-vertebral destruction, CT-paravertebral abscess, MRI-paravertebral abscess, MRI-epidural abscess, postoperative drainage) were selected. Most of the classifiers showed better performance, where the XGBoost model has a higher AUC value (0.86) and lower Brier Score (0.126). The XGBoost model was chosen as the optimal model. The results obtained from the calibration and decision curve analysis (DCA) plots demonstrate that XGBoost has achieved promising performance. After conducting tenfold cross-validation, the XGBoost model demonstrated a mean AUC of 0.85 ± 0.09. SHAP and LIME were used to display the variables’ contributions to the predicted value. The stacked bar plots indicated that infusion volume was the primary contributor, as determined by Gini, permutation importance (PFI), and the LIME algorithm.ConclusionsOur methods not only effectively predicted extended PLOS but also identified risk factors that can be utilized for future treatments. The XGBoost model developed in this study is easily accessible through the deployed web application and can aid in clinical research.

Machine learning for the prediction of 1-year mortality in patients with sepsis-associated acute kidney injury

IntroductionSepsis-associated acute kidney injury (SA-AKI) is strongly associated with poor prognosis. We aimed to build a machine learning (ML)-based clinical model to predict 1-year mortality in patients with SA-AKI.MethodsSix ML algorithms were included to perform model fitting. Feature selection was based on the feature importance evaluated by the SHapley Additive exPlanations (SHAP) values. Area under the receiver operating characteristic curve (AUROC) was used to evaluate the discriminatory ability of the prediction model. Calibration curve and Brier score were employed to assess the calibrated ability. Our ML-based prediction models were validated both internally and externally.ResultsA total of 12,750 patients with SA-AKI and 55 features were included to build the prediction models. We identified the top 10 predictors including age, ICU stay and GCS score based on the feature importance. Among the six ML algorithms, the CatBoost showed the best prediction performance with an AUROC of 0.813 and Brier score of 0.119. In the external validation set, the predictive value remained favorable (AUROC = 0.784).ConclusionIn this study, we developed and validated a ML-based prediction model based on 10 commonly used clinical features which could accurately and early identify the individuals at high-risk of long-term mortality in patients with SA-AKI.

Data-driven models to predict the water-to-cement ratio and initial setting time of cement grouts

ABSTRACT As a common defect, the water–cement ratio of grout can currently be monitored only during the maintenance phase, which limits the repair methods and misses the optimal opportunity for repairs. To overcome this limitation, this study integrated ultrasonic parameters previously used to characterise cement-based materials and developed a new Initial Setting Time and Water–Cement Ratio (IST_WCR) risk model to predict the setting time and water–cement ratio grout using machine learning (ML) algorithms. Experiments on grout involved four different water–cement ratios, ranging from 0.11 to 0.18. A data-driven method based on ML was used to extract predictive factors from eight ultrasonic parameters, including the speed, energy, main frequency, and main frequency amplitude of P-waves and S-waves, and to evaluate multiple ML classifiers to establish the IST_WCR risk prediction model. This model underwent internal and external cross-validations and demonstrated very strong performance with a Brier score of under 0.01. The dataset for ML classifiers contained a total of 956 signals and 7648 features. Compared with traditional methods, this method can automatically characterise the setting process of grout and identify defective water–cement ratios at a very early stage of curing.

The diagnostic and triage accuracy of the GPT-3 artificial intelligence model: an observational study

Artificial intelligence (AI) applications in health care have been effective in many areas of medicine, but they are often trained for a single task using labelled data, making deployment and generalisability challenging. How well a general-purpose AI language model performs diagnosis and triage relative to physicians and laypeople is not well understood. We compared the predictive accuracy of Generative Pre-trained Transformer 3 (GPT-3)'s diagnostic and triage ability for 48 validated synthetic case vignettes (<50 words; sixth-grade reading level or below) of both common (eg, viral illness) and severe (eg, heart attack) conditions to a nationally representative sample of 5000 lay people from the USA who could use the internet to find the correct options and 21 practising physicians at Harvard Medical School. There were 12 vignettes for each of four triage categories: emergent, within one day, within 1 week, and self-care. The correct diagnosis and triage category (ie, ground truth) for each vignette was determined by two general internists at Harvard Medical School. For each vignette, human respondents and GPT-3 were prompted to list diagnoses in order of likelihood, and the vignette was marked as correct if the ground-truth diagnosis was in the top three of the listed diagnoses. For triage accuracy, we examined whether the human respondents' and GPT-3's selected triage was exactly correct according to the four triage categories, or matched a dichotomised triage variable (emergent or within 1 day vs within 1 week or self-care). We estimated GPT-3's diagnostic and triage confidence on a given vignette using a modified bootstrap resampling procedure, and examined how well calibrated GPT-3's confidence was by computing calibration curves and Brier scores. We also performed subgroup analysis by case acuity, and an error analysis for triage advice to characterise how its advice might affect patients using this tool to decide if they should seek medical care immediately. Among all cases, GPT-3 replied with the correct diagnosis in its top three for 88% (42/48, 95% CI 75-94) of cases, compared with 54% (2700/5000, 53-55) for lay individuals (p<0.0001) and 96% (637/666, 94-97) for physicians (p=0·012). GPT-3 triaged 70% correct (34/48, 57-82) versus 74% (3706/5000, 73-75; p=0.60) for lay individuals and 91% (608/666, 89-93%; p<0.0001) for physicians. As measured by the Brier score, GPT-3 confidence in its top prediction was reasonably well calibrated for diagnosis (Brier score=0·18) and triage (Brier score=0·22). We observed an inverse relationship between case acuity and GPT-3 accuracy (p<0·0001) with a fitted trend line of -8·33% decrease in accuracy for every level of increase in case acuity. For triage error analysis, GPT-3 deprioritised truly emergent cases in seven instances. A general-purpose AI language model without any content-specific training could perform diagnosis at levels close to, but below, physicians and better than lay individuals. We found that GPT-3's performance was inferior to physicians for triage, sometimes by a large margin, and its performance was closer to that of lay individuals. Although the diagnostic performance of GPT-3 was comparable to physicians, it was significantly better than a typical person using a search engine. The National Heart, Lung, and Blood Institute.

An ensemble machine learning model for predicting one-year mortality in elderly coronary heart disease patients with anemia

ObjectiveThis study was designed to develop and validate a robust predictive model for one-year mortality in elderly coronary heart disease (CHD) patients with anemia using machine learning methods.MethodsDemographics, tests, comorbidities, and drugs were collected for a cohort of 974 elderly patients with CHD. A prospective analysis was performed to evaluate predictive performances of the developed models. External validation of models was performed in a series of 112 elderly CHD patients with anemia.ResultsThe overall one-year mortality was 43.6%. Risk factors included heart rate, chronic heart failure, tachycardia and β receptor blockers. Protective factors included hemoglobin, albumin, high density lipoprotein cholesterol, estimated glomerular filtration rate (eGFR), left ventricular ejection fraction (LVEF), aspirin, clopidogrel, calcium channel blockers, angiotensin converting enzyme inhibitors (ACEIs)/angiotensin receptor blockers (ARBs), and statins. Compared with other algorithms, an ensemble machine learning model performed the best with area under the curve (95% confidence interval) being 0.828 (0.805–0.870) and Brier score being 0.170. Calibration and density curves further confirmed favorable predicted probability and discriminative ability of an ensemble machine learning model. External validation of Ensemble Model also exhibited good performance with area under the curve (95% confidence interval) being 0.825 (0.734–0.916) and Brier score being 0.185. Patients in the high-risk group had more than six-fold probability of one-year mortality compared with those in the low-risk group (P < 0.001). Shaley Additive exPlanation identified the top five risk factors that associated with one-year mortality were hemoglobin, albumin, eGFR, LVEF, and ACEIs/ARBs.ConclusionsThis model identifies key risk factors and protective factors, providing valuable insights for improving risk assessment, informing clinical decision-making and performing targeted interventions. It outperforms other algorithms with predictive performance and provides significant opportunities for personalized risk mitigation strategies, with clinical implications for improving patient care.

Development and validation of a predictive model for postpartum endometritis.

The aim was to develop a predictive tool for anticipating postpartum endometritis occurrences and to devise strategies for prevention and control. Employing a retrospective approach, the baseline data of 200 women diagnosed with postpartum endometritis in a tertiary maternity hospital in Zhejiang Province, spanning from February 2020 to September 2022, was examined. Simultaneously, the baseline data of 1,000 women without endometritis during the same period were explored with a 1:5 ratio. Subsequently, 1,200 women were randomly allocated into a training group dataset and a test group dataset, adhering to a 7:3 split. The selection of risk factors for postpartum endometritis involved employing random forests, lasso regression, and traditional univariate and multifactor logistic regression on the training group dataset. A nomogram was then constructed based on these factors. The model's performance was assessed using the area under the curve (AUC), calculated through plotting the receiver operating characteristic (ROC) curve. Additionally, the Brier score was employed to evaluate the model with a calibration curve. To gauge the utility of the nomogram, a clinical impact curve (CIC) analysis was conducted. This comprehensive approach not only involved identifying risk factors but also included a visual representation (nomogram) and thorough evaluation metrics, ensuring a robust tool for predicting, preventing, and controlling postpartum endometritis. In the multivariate analysis, six factors were identified as being associated with the occurrence of maternal endometritis in the postpartum period. These factors include the number of negative finger tests (OR: 1.159; 95%CI: 1.091-1.233; P < 0.05), postpartum hemorrhage (1.003; 1.002-1.005; P < 0.05), pre-eclampsia (9.769; 4.64-21.155; P < 0.05), maternity methods (2.083; 1.187-3.7; P < 0.001), prenatal reproductive tract culture (2.219; 1.411-3.47; P < 0.05), and uterine exploration (0.441; 0.233-0.803; P < 0.001).A nomogram was then constructed based on these factors, and its predictive performance was assessed using the area under the curve (AUC). The results in both the training group data (AUC: 0.803) and the test group data (AUC: 0.788) demonstrated a good predictive value. The clinical impact curve (CIC) further highlighted the clinical utility of the nomogram. The development of an individualized nomogram for postpartum endometritis infection holds promise for doctors in screening high-risk women, enabling early intervention and ultimately reducing the rate of postpartum endometritis infection. This comprehensive approach, integrating key risk factors and predictive tools, enhances the potential for timely and targeted medical intervention.