Calibration Slope Research Articles

Development and validation of a risk prediction model for worsening renal function in patients with incident heart failure

Abstract Background Worsening renal function (WRF) is one of the strongest predictors of outcome in patients with heart failure (HF). The presence of WRF often influences the decision to start, up-titrate, or discontinue disease modifying HF therapies and is one of the key determinants of suboptimal guideline-directed medical therapy. Existing WRF risk scores were developed in hospitalised HF patients to predict in-hospital WRF. Their performance among incident HF is far from ideal. It is therefore important to develop a new WRF risk prediction model in people with newly diagnosed HF. Purpose This study was to develop and validate a clinical risk prediction model for 180-day WRF post diagnosis of incident HF using data from a population-based longitudinal cohort. Methods We developed and validated a multivariable logistic regression model for WRF in the 180 days post diagnosis of incident HF. Data for individuals with incident HF between 2016 to 2021 were extracted from the NELSON study conducted in Tayside, Scotland. WRF was defined as a composite event of (i) having two consecutive eGFRs declined by 40% or greater; (ii) having an eGFR < 15mL/min/1.73m2; (iii) initiation of sustained dialysis; (iv) development of end-stage kidney disease; (v) receiving kidney transplantation; or (vi) kidney related death. Four candidate models included one model using literature knowledge; one multivariable fractional polynomial (MFP) model; and two stepwise selected models based on either Akaike or Bayesian information criterion (AIC or BIC). Calibration and discrimination were assessed, and performance stability were evaluated using optimism corrected performance via 1000 times bootstrapping. Results 4076 individuals (mean age 72.8 ± 13.2; 58% male; 1081 HF with reduced ejection fraction (HFrEF), 646 with HF with mildly reduced ejection fraction (HFmrEF), 1348 HF with preserved ejection fraction (HFpEF), and 1001 HF with unknown ejection fraction (EF)) who were WRF-free when diagnosed with HF were identified. Of these, 2095 (51.4%) were in-patients, and 1981 (48.6%) were out-patients. 285 (7%) patients developed WRF within 180 days post HF diagnosis. The selected MFP model (comprising 12 predictors, Table 1) showed good calibration (slope = 1.00 [0.86, 1.14]; intercept = 0.0 [-0.13, 0.13]) discrimination (C-statistic = 0.69 [95% CI: 0.65, 0.72]), and stability (optimism corrected: calibration slope = 0.94 [0.80, 1.07]; calibration intercept = -0.14 [-0.50, 0.20]; C-statistic = 0.67 [0.64, 0.71]). Discrimination of the developed model significantly outperformed two existing risk scores (Forman and Basel, both with a C-statistic = 0.62 [0.59 – 0.65]). Conclusions We have shown that the developed model performed better in predicting WRF in individuals with newly diagnosed HF. This could have utility in identifying at risk patients with HF early in the disease trajectory for therapies that can reduce renal decline.Calibration of predicted riskModel estimates for predicting 180d WRF

Long-term outcomes of patients with apical hypertrophic cardiomyopathy: incorporation of a novel risk score

Abstract Background Apical hypertrophic cardiomyopathy (aHCM) is a subtype of HCM characterized by left ventricular hypertrophy predominantly in the apex. There is still limited data on factors associated with adverse outcomes in patients with aHCM. Objectives To a) describe the characteristics and b) develop a novel risk score associated with outcomes in aHCM patients. Methods A total of 462 patients (mean age 58±15 years, 68% male and 70% white) diagnosed with aHCM based on clinical, echocardiographic, and cardiac magnetic resonance (CMR) evaluation were included. The primary endpoint was all-cause death or aborted sudden cardiac death. Using the cox regression analysis, variables showing potential association with the composite endpoint were considered to develop an aHCM specific risk score. Results On echocardiography, the mean LV ejection fraction (LVEF) was 64±8%, and the mean LV mass was 223±72 g. A baseline CMR was performed in 246 (53%) patients, with delayed gadolinium enhancement in 170 (71%) patients with a mean percentage of 5±7%. Patients were followed-up for a mean of 68 months (range 1-248), with composite events occurring in 80 (17%) patients (death in 62 [13%] patients). Univariate Cox regression analysis identified age hazard ratio (HR) 1.06 (95% Confidence Interval or CI 1.04-1.08; p< 0.001), apical aneurysm HR 2.32 (95% CI 1.24-4.34 p=0.008), lower LVEF HR 0.96 (95% CI 0.94-0.99 p=0.002), higher right ventricular systolic pressure (RVSP) HR 1.04 (95% CI 1.02-1.06 p< 0.001), New York Heart Association Class IV HR 11.69 (95% CI 3.59-38.08 p< 0.001) and the presence of Atrial fibrillation HR 1.77 (95% CI 1.14-2.76 p=0.012). An aHCM specific risk score was subsequently created (Figure 1), with good discrimination (c-statistic=0.75) and good calibration with expected to observed ratio of 1.02 and a calibration slope of 0.91 (p-value for difference between expected and observed probabilities =0.22). The risk score ranges between 0 and 8 points, with higher score associated with a higher rate of composite events (Figure 2). Conclusion We developed a novel risk score which demonstrated incremental prognostic value for hard events in aHCM patients.Risk prediction modelKaplan Meier event-free survival curves

Nationwide study on development and validation of a risk prediction model for CIN3+ and cervical cancer in Estonia

Transitioning to an individualized risk-based approach can significantly enhance cervical cancer screening programs. We aimed to derive and internally validate a prediction model for assessing the risk of cervical intraepithelial neoplasia grade 3 or higher (CIN3+) and cancer in women eligible for screening. This retrospective study utilized data from the Estonian electronic health records, including 517,884 women from the health insurance database and linked health registries. We employed Cox proportional hazard regression, incorporating reproductive and medical history variables (14 covariates), and utilized the least absolute shrinkage and selection operator (LASSO) for variable selection. A 10-fold cross-validation for internal validation of the model was used. The main outcomes were the performance of discrimination and calibration. Over the 8-year follow-up, we identified 1326 women with cervical cancer and 5929 with CIN3+, with absolute risks of 0.3% and 1.1%, respectively. The prediction model for CIN3 + and cervical cancer had good discriminative power and was well calibrated Harrell’s C of 0.74 (0.73–0.74) (calibration slope 1.00 (0.97–1.02) and 0.67 (0.66–0.69) (calibration slope 0.92 (0.84–1.00) respectively. A developed model based on nationwide electronic health data showed potential utility for risk stratification to supplement screening efforts. This work was supported through grants number PRG2218 from the Estonian Research Council, and EMP416 from the EEA (European Economic Area) and Norway Grants.

External validation and calibration of the HoLISTIC Consortium's advanced-stage Hodgkin lymphoma international prognostic index (A-HIPI) in the Brazilian Hodgkin lymphoma registry.

The Hodgkin lymphoma International Study for Individual Care (HoLISTIC) Consortium's A-HIPI model, developed in 2022 for advanced-stage classical Hodgkin lymphoma (cHL), predicts survival within 5 years amongst newly diagnosed patients. This study validates its performance in the Brazilian Hodgkin lymphoma registry. By 2022, the Brazilian HL registry included 1357 cHL patients, with a median 5-year follow-up. Probabilities for 5-year progression-free survival (PFS) and overall survival (OS) were calculated using A-HIPI-model equations. Discrimination (Harrell C-statistic/Uno C-statistic) and calibration measures assessed external validation and calibration. Lab values beyond the allowed range were excluded, mirroring the initial A-HIPI analysis. A total of 694 advanced-stage cHL patients met the original inclusion criteria (age 18-65 years, Stage IIB-IV). Median age was 31 years; 46.3% were females. Stage distribution was IIB (33.1%), III (27.4%), IV (39.5%). Bulky disease in 32.6%. Five-year PFS and OS were 68.4% and 86.0%, respectively. Harrell C-statistics were 0.60 for PFS and 0.69 for OS, and Uno C-statistics were 0.63 for PFS and 0.72 for OS. Calibration plots demonstrated well-calibrated predictions with calibration slopes of 0.91 and 1.03 for 5-year OS and PFS, respectively. Despite differing patient, clinical characteristics, and socioeconomic factors, the baseline prediction tool performed well in the Brazilian cohort, demonstrating adequate discrimination and calibration. This supports its reliability in diverse settings.

External validation and calibration of risk equations for prediction of diabetic kidney diseases among patients with type 2 diabetes in Taiwan

BackgroundMost existing risk equations for predicting/stratifying individual diabetic kidney disease (DKD) risks were developed using relatively dated data from selective and homogeneous trial populations comprising predominately Caucasian type 2 diabetes (T2D) patients. We seek to adapt risk equations for prediction of DKD progression (microalbuminuria, macroalbuminuria, and renal failure) using empiric data from a real-world population with T2D in Taiwan.MethodsRisk equations from three well-known simulation models: UKPDS-OM2, RECODe, and CHIME models, were adapted. Discrimination and calibration were determined using the area under the receiver operating characteristic curve (AUROC), a calibration plot (slope and intercept), and the Greenwood-Nam-D’Agostino (GND) test. Recalibration was performed for unsatisfactory calibration (p-value of GND test < 0.05) by adjusting the baseline hazards of risk equations to address risk variations among patients.ResultsThe RECODe equations for microalbuminuria and macroalbuminuria showed moderate discrimination (AUROC: 0.62 and 0.76) but underestimated the event risks (calibration slope > 1). The CHIME equation had the best discrimination for renal failure (AUROCs from CHIME, UKPDS-OM2 and RECODe: 0.77, 0.60 and 0.64, respectively). All three equations overestimated renal failure risk (calibration slope < 1). After rigorous updating, the calibration slope/intercept of the recalibrated RECODe for predicting microalbuminuria (0.87/0.0459) and macroalbuminuria (1.10/0.0004) risks as well as the recalibrated CHIME equation for predicting renal failure risk (0.95/-0.0014) were improved.ConclusionsRisk equations for prediction of DKD progression in real-world Taiwanese T2D patients were established, which can be incorporated into a multi-state simulation model to project and differentiate individual DKD risks for supporting timely interventions and health economic research.

Fracture Risk Prediction Using the Fracture Risk Assessment Tool in Individuals With Cancer

The Fracture Risk Assessment Tool (FRAX) is a fracture risk prediction tool for 10-year probability of major osteoporotic fracture (MOF) and hip fracture in the general population. Whether FRAX is useful in individuals with cancer is uncertain. To determine the performance of FRAX for predicting incident fractures in individuals with cancer. This retrospective population-based cohort study included residents of Manitoba, Canada, with and without cancer diagnoses from 1987 to 2014. Diagnoses were identified through the Manitoba Cancer Registry. Incident fractures to March 31, 2021, were identified in population-based health care data. Data analysis occurred between January and March 2023. FRAX scores were computed for those with bone mineral density (BMD) results that were recorded in the Manitoba BMD Registry. This study included 9877 individuals with cancer (mean [SD] age, 67.1 [11.2] years; 8693 [88.0%] female) and 45 877 individuals in the noncancer cohort (mean [SD] age, 66.2 [10.2] years; 41 656 [90.8%] female). Compared to individuals without cancer, those with cancer had higher rates of incident MOF (14.5 vs 12.9 per 1000 person-years; P < .001) and hip fracture (4.2 vs 3.5 per 1000 person-years; P = .002). In the cancer cohort, FRAX with BMD results were associated with incident MOF (HR per SD increase, 1.84 [95% CI, 1.74-1.95]) and hip fracture (HR per SD increase, 3.61 [95% CI, 3.13-4.15]). In the cancer cohort, calibration slopes for FRAX with BMD were 1.03 for MOFs and 0.97 for hip fractures. In this retrospective cohort study, FRAX with BMD showed good stratification and calibration for predicting incident fractures in patients with cancer. These results suggest that FRAX with BMD can be a reliable tool for predicting incident fractures in individuals with cancer.

Predictive Factors of Apparent Treatment Resistant Hypertension Among Patients With Hypertension Identified Using Electronic Health Records.

Early identification of a patient with resistant hypertension (RH) enables quickly intensified treatment, short-interval follow-up, or perhaps case management to bring his or her blood pressure under control and reduce the risk of complications. To identify predictors of RH among individuals with newly diagnosed hypertension (HTN), while comparing different prediction models and techniques for managing missing covariates using electronic health records data. Risk prediction study in a retrospective cohort. Adult patients with incident HTN treated in any of the primary care clinics of one health system between April 2013 and December 2016. Predicted risk of RH at the time of HTN identification and candidate predictors for variable selection in future model development. Among 26,953 individuals with incident HTN, 613 (2.3%) met criteria for RH after 4.7 months (interquartile range, 1.2-11.3). Variables selected by the least absolute shrinkage and selection operator (LASSO), included baseline systolic blood pressure (SBP) and its missing indicator (a dummy variable created if baseline SBP is absent), use of antihypertensive medication at the time of cohort entry, body mass index, and atherosclerosis risk. The random forest technique achieved the highest area under the curve (AUC) of 0.893 (95% CI, 0.881-0.904) and the best calibration with a calibration slope of 1.01. Complete case analysis is not a valuable option (AUC = 0.625). Machine learning techniques and traditional logistic regression exhibited comparable levels of predictive performance after handling the missingness. We suggest that the variables identified by this study may be good candidates for clinical prediction models to alert clinicians to the need for short-interval follow up and more intensive early therapy for HTN.

Multicentre Validation of the RESECT-90 Prediction Model for 90-Day Mortality After Lung Resection

BackgroundThe RESECT-90 model was developed to predict 90-day mortality for patients undergoing lung resection but hasn't been externally validated. The aim of this study was to validate the RESECT-90 clinical prediction model using multicentre patient data from across the United Kingdom (UK). Materials and MethodsData from 12 UK thoracic surgery centers for patients undergoing lung resection between 2016 and 2020 with available 90-day mortality status were used to externally validate the RESECT-90 model. Measures of discrimination (area under the receiving operator characteristic curve [AUC]) and calibration (calibration slope, calibration intercept and flexible calibration plot) were assessed as measures of model performance. Model recalibration was also performed by updating the original model intercept and coefficients. ResultsA total of 12,241 patients were included. Overall 90-day mortality was 2.9% (n = 360). Acceptable model discrimination was demonstrated (AUC 0.74 [0.73, 0.75]). Calibration varied between centers with some evidence of overall model miscalibration (calibration slope 0.80 [0.66, 0.95] and calibration intercept 0.40 [0.29, 0.52]) despite acceptable appearances of the flexible calibration plot. The model was subsequently recalibrated, after which all measures of calibration indicated excellent performance. ConclusionsAfter external validation and recalibration using a large contemporary cohort of patients undergoing surgery in multiple geographical locations across the UK, the RESECT-90 model demonstrated satisfactory statistical performance for the prediction of 90-day mortality after lung resection. Whilst the recalibrated model will require ongoing validation, the results of this study suggest that routine use of the RESECT-90 model in UK thoracic surgery practice should be considered.

External Validation of the American Heart Association PREVENT Cardiovascular Disease Risk Equations

The American Heart Association's Predicting Risk of Cardiovascular Disease Events (PREVENT) equations were developed to extend and improve on previous cardiovascular disease (CVD) risk assessments for the purpose of treatment initiation and patient-clinician communication. To assess prognostic capabilities, calibration, and discrimination of the PREVENT equations in a study sample representative of the noninstitutionalized, US general population. This prognostic study used data from the National Health and Nutrition Examination Survey (NHANES) 1999 to 2010 data cycles. Participants included adults for whom 10-year follow-up data were available. Data curation and analyses took place from December 2023 through May 2024. Primary measures were risk estimated by the PREVENT equations, as well as risk estimates from the previous Pooled Cohort Equations (PCEs). The primary outcome was composite CVD-related mortality at 10 years of follow-up. Additional analyses compared the PREVENT equations against the PCEs. Model discrimination was assessed with receiver-operator characteristic curves and Harrell C statistic from proportional hazard regression; model calibration was determined as the slope of predicted versus observed risk. The study cohort, accounting for NHANES complex survey design, consisted of 172.9 million participants (mean age, 45.0 years [95% CI, 44.6-45.4 years]; 52.1% women [95% CI, 51.5%-52.6%]). In analyses adjusted for the NHANES survey design, a 1% increase in PREVENT risk estimates was statistically significantly associated with increased CVD mortality risk (hazard ratio, 1.090; 95% CI, 1.087-1.094). PREVENT risk scores demonstrated excellent discrimination (C statistic, 0.890; 95% CI, 0.881-0.898) but moderate underfitting of the model (calibration slope, 1.13; 95% CI, 1.06-1.21). PREVENT risk models performed statistically significantly better than the PCEs, as assessed by the net reclassification index (0.093; 95% CI, 0.073-0.115). In this prognostic study of the PREVENT equations, PREVENT risk estimates demonstrated excellent discrimination and only modest discrepancies in calibration. These findings provided evidence supporting utilization of the PREVENT equations for application in the intended population as suggested by the American Heart Association.

Development and validation of a prognostic model to predict relapse in adults with remitted depression in primary care: secondary analysis of pooled individual participant data from multiple studies.

Relapse of depression is common and contributes to the overall associated morbidity and burden. We lack evidence-based tools to estimate an individual's risk of relapse after treatment in primary care, which may help us more effectively target relapse prevention. The objective was to develop and validate a prognostic model to predict risk of relapse of depression in primary care. Multilevel logistic regression models were developed, using individual participant data from seven primary care-based studies (n=1244), to predict relapse of depression. The model was internally validated using bootstrapping, and generalisability was explored using internal-external cross-validation. Residual depressive symptoms (OR: 1.13 (95% CI: 1.07 to 1.20), p<0.001) and baseline depression severity (OR: 1.07 (1.04 to 1.11), p<0.001) were associated with relapse. The validated model had low discrimination (C-statistic 0.60 (0.55-0.65)) and miscalibration concerns (calibration slope 0.81 (0.31-1.31)). On secondary analysis, being in a relationship was associated with reduced risk of relapse (OR: 0.43 (0.28-0.67), p<0.001); this remained statistically significant after correction for multiple significance testing. We could not predict risk of depression relapse with sufficient accuracy in primary care data, using routinely recorded measures. Relationship status warrants further research to explore its role as a prognostic factor for relapse. Until we can accurately stratify patients according to risk of relapse, a universal approach to relapse prevention may be most beneficial, either during acute-phase treatment or post remission. Where possible, this could be guided by the presence or absence of known prognostic factors (eg, residual depressive symptoms) and targeted towards these. NCT04666662.

Understanding overfitting in random forest for probability estimation: a visualization and simulation study

BackgroundRandom forests have become popular for clinical risk prediction modeling. In a case study on predicting ovarian malignancy, we observed training AUCs close to 1. Although this suggests overfitting, performance was competitive on test data. We aimed to understand the behavior of random forests for probability estimation by (1) visualizing data space in three real-world case studies and (2) a simulation study.MethodsFor the case studies, multinomial risk estimates were visualized using heatmaps in a 2-dimensional subspace. The simulation study included 48 logistic data-generating mechanisms (DGM), varying the predictor distribution, the number of predictors, the correlation between predictors, the true AUC, and the strength of true predictors. For each DGM, 1000 training datasets of size 200 or 4000 with binary outcomes were simulated, and random forest models were trained with minimum node size 2 or 20 using the ranger R package, resulting in 192 scenarios in total. Model performance was evaluated on large test datasets (N = 100,000).ResultsThe visualizations suggested that the model learned “spikes of probability” around events in the training set. A cluster of events created a bigger peak or plateau (signal), isolated events local peaks (noise). In the simulation study, median training AUCs were between 0.97 and 1 unless there were 4 binary predictors or 16 binary predictors with a minimum node size of 20. The median discrimination loss, i.e., the difference between the median test AUC and the true AUC, was 0.025 (range 0.00 to 0.13). Median training AUCs had Spearman correlations of around 0.70 with discrimination loss. Median test AUCs were higher with higher events per variable, higher minimum node size, and binary predictors. Median training calibration slopes were always above 1 and were not correlated with median test slopes across scenarios (Spearman correlation − 0.11). Median test slopes were higher with higher true AUC, higher minimum node size, and higher sample size.ConclusionsRandom forests learn local probability peaks that often yield near perfect training AUCs without strongly affecting AUCs on test data. When the aim is probability estimation, the simulation results go against the common recommendation to use fully grown trees in random forest models.

SACrA score to predict the initiation of renal replacement therapy in critically ill patients: a single-center retrospective study

Background Acute kidney injury (AKI) is a prevalent complication in critically ill patients that affects the timing of renal replacement therapy (RRT) initiation. This study aimed to develop and validate the SACrA score for predicting non-emergent initiations (BUN ≥112 mg/dL or oliguria for >72 h) of RRT in critically ill patients. Methods We conducted a retrospective cohort study using data from two cohorts. The derivation cohort included patients admitted to the ICU between November 2021 and December 2023, whereas the validation cohort included patients admitted between September 2019 and October 2021. The primary outcome was non-emergent RRT initiation. The multivariate logistic regression with stepwise selection based on the Akaike information criterion finalized the model, including the variables, such as sex, albumin (Alb), creatinine (Cr), and APACHE II score (SACrA). Results The derivation and validation cohorts comprised 470 and 476 patients, respectively. The SACrA score showed a strong predictive performance for non-emergent RRT initiation in both the cohorts. Cohort 1 had an ROC–AUC of 0.971, with a calibration slope of 0.982 and an intercept of 0.009, whereas cohort 2 had an ROC–AUC of 0.918, with a calibration slope of 0.988 and an intercept of 0.004. Conclusions The SACrA score is a robust tool for predicting non-emergent RRT initiation in critically ill patients using readily available clinical variables. Though additional data are needed to validate the SACrA score, our analysis suggests the tool may help clinicians make informed decisions, reduce unnecessary RRT, and thereby improve patient outcomes.

Predicting extended hospital stay following revision total hip arthroplasty: a machine learning model analysis based on the ACS-NSQIP database.

Prolonged length of stay (LOS) following revision total hip arthroplasty (THA) can lead to increased healthcare costs, higher rates of readmission, and lower patient satisfaction. In this study, we investigated the predictive power of machine learning (ML) models for prolonged LOS after revision THA using patient data from a national-scale patient repository. We identified 11,737 revision THA cases from the American College of Surgeons National Surgical Quality Improvement Program database from 2013 to 2020. Prolonged LOS was defined as exceeding the 75th value of all LOSs in the study cohort. We developed four ML models: artificial neural network (ANN), random forest, histogram-based gradient boosting, and k-nearest neighbor, to predict prolonged LOS after revision THA. Each model's performance was assessed during training and testing sessions in terms of discrimination, calibration, and clinical utility. The ANN model was the most accurate with an AUC of 0.82, calibration slope of 0.90, calibration intercept of 0.02, and Brier score of 0.140 during testing, indicating the model's competency in distinguishing patients subject to prolonged LOS with minimal prediction error. All models showed clinical utility by producing net benefits in the decision curve analyses. The most significant predictors of prolonged LOS were preoperative blood tests (hematocrit, platelet count, and leukocyte count), preoperative transfusion, operation time, indications for revision THA (infection), and age. Our study demonstrated that the ML model accurately predicted prolonged LOS after revision THA. The results highlighted the importance of the indications for revision surgery in determining the risk of prolonged LOS. With the model's aid, clinicians can stratify individual patients based on key factors, improve care coordination and discharge planning for those at risk of prolonged LOS, and increase cost efficiency.

Validation of the BOADICEA model for epithelial tubo-ovarian cancer risk prediction in UK Biobank

BackgroundThe clinical validity of the multifactorial BOADICEA model for epithelial tubo-ovarian cancer (EOC) risk prediction has not been assessed in a large sample size or over a longer term.MethodsWe evaluated the model discrimination and calibration in the UK Biobank cohort comprising 199,429 women (733 incident EOCs) of European ancestry without previous cancer history. We predicted 10-year EOC risk incorporating data on questionnaire-based risk factors (QRFs), family history, a 36-SNP polygenic risk score and pathogenic variants (PV) in six EOC susceptibility genes (BRCA1, BRCA2, RAD51C, RAD51D, BRIP1 and PALB2).ResultsDiscriminative ability was maximised under the multifactorial model that included all risk factors (AUC = 0.68, 95% CI: 0.66–0.70). This model was well calibrated in deciles of predicted risk with calibration slope=0.99 (95% CI: 0.98–1.01). Discriminative ability was similar in women younger or older than 60 years. The AUC was higher when analyses were restricted to PV carriers (0.76, 95% CI: 0.69–0.82). Using relative risk (RR) thresholds, the full model classified 97.7%, 1.7%, 0.4% and 0.2% women in the RR < 2.0, 2.0 ≤ RR < 2.9, 2.9 ≤ RR < 6.0 and RR ≥ 6.0 categories, respectively, identifying 9.1 of incident EOC among those with RR ≥ 2.0.DiscussionBOADICEA, implemented in CanRisk (www.canrisk.org), provides valid 10-year EOC risks and can facilitate clinical decision-making in EOC risk management.

External validation of the BE-ALIVE score for predicting 30-day mortality in patients presenting with acute coronary syndromes

IntroductionThe BE-ALIVE score is an additive scoring system for estimating 30-day mortality in patients presenting with an acute coronary syndrome (ACS) [1]. However, it had only previously been tested on an internal validation cohort. The aim was to assess the scoring system on an external validation cohort. MethodsThe scoring system comprises six domains: (1) Base Excess (1 point for < −2 mmols/L), (2) Age (<65 years: 0 points, 65–74: 1 point, 75–84: 2 points, ≥ 85: 3 points), (3) Lactate (<2 mmols/L: 0 points, 2–4.9: 1 point, 5–9.9: 3 points, ≥ 10: 6 points), (4) Intubated & Ventilated (2 points), (5) Left Ventricular function (normal or mildly impaired: -1 point, moderately impaired: 1 point, severely impaired: 3 points) and (6) External / out of hospital cardiac arrest (1 point).We applied the BE-ALIVE score was applied to 205 consecutive patients at a different institution. ResultsCalibration was strong, with an observed to expected ratio of 1.01, a calibration slope of 1.26 and calibration in the large of −0.03. The Spiegelhalter's Z-statistic was −0.95 (p = 0.34). The AUC was 0.95 (0.92–0.98) in the external validation cohort versus 0.90 (0.85–0.95) during internal validation.Overall performance was excellent with a Brier score of 0.07 versus 0.06 during internal validation. The negative predictive value for 30-day mortality of a BE-ALIVE score < 4 was 98 %, with a positive predicted value of a score ≥ 10 of 95 %. ConclusionsThe BE-ALIVE score remains a robust predictor of 30-day mortality in an external validation cohort.

Risk-scoring system predicting need for hospital-specific interventional care after peroral endoscopic myotomy.

Early identification of patients needing hospital-specific interventional care (HIC) following endoscopic treatment is valuable for optimizing postoperative hospital stays. We aimed to develop and validate a risk-scoring system for predicting HIC in patients who underwent peroral endoscopic myotomy (POEM). This study included patients with esophageal motility disorders who underwent POEM at our hospital between April 2015 and March 2023. HIC was defined as any of the following situations: fasting for gastrointestinal rest to manage adverse events (AEs); intravenous administration of medications such as antibiotics and blood transfusion; endoscopic, radiologic, and surgical interventions; intensive care unit management; or other life-threatening events. A risk-scoring system for predicting HIC after postoperative day (POD) 1 was developed using multivariable logistic regression and was internally validated using bootstrapping and decision curve analysis. Of the 589 patients, 50 (8.5%) experienced HIC after POD1. Risk scores were assigned for four factors as follows: age (0 points for <70 years, 1 point for 70-79 years, 2 points for ≥80 years), preoperative prognostic nutritional index (0 points for >45, 1 point for 40-45, 4 points for <40), postoperative surgical site AEs on second-look endoscopy (7 points), and postoperative pneumonia on chest radiography (6 points). The discriminative ability (concordance statistics, 0.85; 95% confidence interval, 0.78-0.91) and calibration (slope 1.00; 0.74-1.28) were satisfactory. The decision curve analysis demonstrated its clinical usefulness. This risk-scoring system can predict the HIC after POD1 and provide useful information for determining discharge.

A Multicenter Validation of a Novel Prediction Model for Elbow Flexion Recovery after Nerve Transfer Surgery in Brachial Plexus Injuries.

Nerve transfer surgery for brachial plexus injuries exhibits variable success rates, potentially resulting in prolonged limb dysfunction for more than 2 years. A proposed prediction model has been developed to predict the unsuccessful recovery of elbow flexion after the surgery. The model consisted of six variables, namely body mass index 23 kg/m2 or more, smoking, total arm type, donor nerve, ipsilateral upper extremity fracture, and ipsilateral vascular injury. This study aimed to assess the external validity of the model for wider applicability. This retrospective analysis examined the medical records of 213 eligible patients with traumatic brachial plexus injuries who underwent surgery at two referral centers between July 2008 and June 2022. The prediction model was applied to estimate recovery failure probability, which was compared with the observed outcomes for each patient. Both the original and simplified models were validated for discrimination and calibration using metrics including c-statistic, Hosmer-Lemeshow goodness-of-fit test, calibration plot, calibration slope, and intercept. Thirty-two percent of patients experienced unsuccessful elbow flexion recovery. Both the original and simplified models demonstrated good discrimination (c-statistics: 0.748 and 0.759, respectively). The Hosmer-Lemeshow test revealed strong agreement between predicted and observed probabilities for both models (P = 0.66 and P = 0.92, respectively). The calibration plot exhibited good agreement, with a calibration slope of 0.928 and an intercept of 0.377. The prediction model showed strong external validation, confirming its clinical value. High-risk patients should be educated on the risks and benefits of nerve transfer surgery and consider alternative treatments such as primary free functioning muscle transfer.

The performance of prognostic models depended on the choice of missing value imputation algorithm: a simulation study

ObjectiveThe development of clinical prediction models is often impeded by the occurrence of missing values in the predictors. Various methods for imputing missing values before modelling have been proposed. Some of them are based on variants of multiple imputation by chained equations, while others are based on single imputation. These methods may include elements of flexible modelling or machine learning algorithms, and for some of them user-friendly software packages are available. The aim of this study was to investigate by simulation if some of these methods consistently outperform others in performance measures of clinical prediction models. Study Design and SettingWe simulated development and validation cohorts by mimicking observed distributions of predictors and outcome variable of a real data set. In the development cohorts, missing predictor values were created in 36 scenarios defined by the missingness mechanism and proportion of non-complete cases. We applied three imputation algorithms that were available in R software: mice, aregImpute and missForest. These algorithms differed in their use of linear or flexible models, or random forests, the way of sampling from the predictive posterior distribution, and the generation of a single or multiple imputed data sets. For multiple imputation we also investigated the impact of the number of imputations. Logistic regression models were fitted with the simulated development cohorts before (full data analysis) and after missing value generation (complete case analysis), and with the imputed data. Prognostic model performance was measured by the scaled Brier score, c-statistic, calibration intercept and slope, and by the mean absolute prediction error evaluated in validation cohorts without missing values. Performance of full data analysis was considered as ideal. ResultsNone of the imputation methods achieved the model’s predictive accuracy that would be obtained in case of no missingness. In general, complete case analysis yielded the worst performance, and deviation from ideal performance increased with increasing percentage of missingness and decreasing sample size. Across all scenarios and performance measures, aregImpute and mice, both with 100 imputations, resulted in highest predictive accuracy. Surprisingly aregImpute outperformed full data analysis in achieving calibration slopes very close to 1 across all scenarios and outcome models. The increase of mice’s performance with 100 compared to 5 imputations was only marginal. The differences between the imputation methods decreased with increasing sample sizes and decreasing proportion of non-complete cases. ConclusionIn our simulation study, model calibration was more affected by the choice of the imputation method than model discrimination. While differences in model performance after using imputation methods were generally small, multiple imputation methods as mice and aregImpute that can handle linear or nonlinear associations between predictors and outcome are an attractive and reliable choice in most situations.

Factors related to the occurrence of fetal birth defects and the construction of a Nomogram model.

To explore the influencing factors of fetal birth defects (BD) and construct a nomogram model. A total of 341 newborns admitted to Meizhou people's hospital from September 2021 to September 2023 were randomly grouped into a modeling group (239 cases) and a validation group (102 cases). The modeling group fetuses were separated into BD and non-BD groups. Multivariate logistic regression analyzed risk factors for BD; R software constructed a nomogram model; Receiver operating characteristic (ROC) curve evaluated the model's discrimination for BD. The top 5 types of BD were congenital heart disease, polydactyly/syndactyly, cleft lip/palate, ear malformation, and foot malformation, with incidence rates of 23.81%, 20.63%, 12.70%, 11.11%, and 7.94%, respectively. BD incidence was 26.36% (63/239). Significant differences between BD and non-BD groups were found in maternal age, gestational age, history of adverse pregnancy/childbirth, gestational hypertension, adverse emotions during pregnancy, and folic acid intake duration (P<0.05). Logistic regression showed maternal age (OR: 4.125), gestational age (OR: 3.066), adverse pregnancy history (OR: 10.628), gestational hypertension (OR: 5.658), adverse emotions (OR: 5.467), and folic acid intake duration (OR: 4.586) were risk factors for BD (P<0.05). The modeling group's ROC AUC was 0.938, calibration curve slope close to 1, H-L test =8.342, P=0.692; external validation AUC was 0.961, calibration slope close to 1, H-L test =7.634, P=0.635. Identified risk factors include maternal age, gestational age, adverse pregnancy history, gestational hypertension, adverse emotions, and folic acid intake duration. The nomogram model shows good discrimination and consistency for evaluating neonatal BD risk.

Long-Term Outcomes of Patients With Apical Hypertrophic Cardiomyopathy Utilizing a New Risk Score

BackgroundApical hypertrophic cardiomyopathy (aHCM) is a distinct variant characterized by predominant hypertrophy of the left ventricle apex. ObjectivesThis study sought to describe aHCM patients' characteristics and develop a risk score for aHCM patients. MethodsA total of 462 patients (age 58 ± 15 years, 68% male) diagnosed with aHCM were included. The primary endpoint was death, appropriate defibrillator discharge, or need for cardiac transplantation. Variables showing potential association with the composite endpoint were considered to develop an aHCM-specific risk score. ResultsAt baseline, 67% patients were asymptomatic and 69% had no risk factors for sudden death. On echocardiography, the mean left ventricle ejection fraction, left atrial volume index, and right ventricular systolic pressure were 64% ± 8%, 36 ± 15 ml/m2, and 32 ± 10 mm Hg, respectively, with 51(11%) demonstrating an apical aneurysm. Baseline cardiac magnetic resonance, performed in 246 (53%) patients, demonstrated delayed gadolinium enhancement in 170 (71%) patients (mean percentage of 4.9% ± 6.6%). At age 6.3 ± 4.8 years, the composite events occurred in 80 (17%, death in 62 [13%]) patients. The aHCM-specific risk score, incorporating age, apical aneurysm, left atrial volume index, serum creatinine, and right ventricular systolic pressure, demonstrated good discrimination (C-statistic = 0.75) with an expected to observed ratio of 1.02 and a calibration slope of 0.91. The risk score ranged between 0 and 8 points, with a higher score associated with higher composite events. ConclusionsaHCM constituted 6.8% of our overall HCM cohort with a composite event rate of 2.8%/year. The aHCM risk score provided good discrimination in predicting the composite primary endpoint, with a higher score associated with a higher rate of events.