Performance Of Risk Models Research Articles

Polygenic Risk Scores in Human Disease

Abstract Background Polygenic risk scores (PRS) are measures of genetic susceptibility to human health traits. With the advent of large data repositories combining genetic data and phenotypic information, PRS are providing valuable insights into the genetic architecture of complex diseases and are transforming the landscape of precision medicine. Content PRS have emerged as tools with clinical utility in human disease. Herein, details on how to develop PRS are provided, followed by 5 areas in which they can be used to improve human health: (a) augmenting risk prediction, (b) refining diagnosis, (c) guiding treatment choices, (d) making clinical trials more efficient, and (e) improving public health. Finally, some of the ongoing challenges to the clinical implementation of PRS are noted. Summary PRS can offer valuable information for providers and patients, including identifying risk of disease earlier in life and before the onset of clinical risk factors, guiding treatment decisions, improving public health outcomes, and making clinical trials more efficient. The future of genomic-informed risk assessments of disease is through integrated risk models that combine genetic factors including PRS, monogenic, and somatic DNA information with nongenetic risk factors such as clinical risk estimators and multiomic data. However, adopting PRS in a clinical setting at scale faces some challenges, including cross-ancestry performance, standardization and calibration of risk models, downstream clinical decision-making from risk information, and seamless integration into existing health systems.

Performance of Risk Models for Antimicrobial Resistance in Adult Patients With Sepsis

The results of prediction models that stratify patients with sepsis and risk of resistant gram-negative bacilli (GNB) infections inform treatment guidelines. However, these models do not extrapolate well across hospitals. To assess whether patient case mix and local prevalence rates of resistance contributed to the variable performance of a general risk stratification GNB sepsis model for community-onset and hospital-onset sepsis across hospitals. This was a retrospective cohort study conducted from January 2016 and October 2021. Adult patients with sepsis at 10 acute-care hospitals in rural and urban areas across Missouri and Illinois were included. Inclusion criteria were blood cultures indicating sepsis, having received 4 days of antibiotic treatment, and having organ dysfunction (vasopressor use, mechanical ventilation, increased creatinine or bilirubin levels, and thrombocytopenia). Analyses were completed in April 2024. The model included demographic characteristics, comorbidities, vital signs, laboratory values, procedures, and medications administered. Culture results were stratified for ceftriaxone-susceptible GNB (SS), ceftriaxone-resistant but cefepime-susceptible GNB (RS), and ceftriaxone- and cefepime-resistant GNB (RR). Negative cultures and other pathogens were labeled SS. Deep learning models were developed separately for community-onset (patient presented with sepsis) and hospital-onset (sepsis developed ≥48 hours after admission) sepsis. The models were tested across hospitals and patient subgroups. Models were assessed using area under the receiver operating characteristic curve (AUROC) and area under precision recall curve (AUPRC). A total of 39 893 patients with 85 238 sepsis episodes (43 207 [50.7%] community onset; 42 031 [48.3%] hospital onset) were included. Median (IQR) age was 65 (54-74) years, 21 241 patients (53.2%) were male, and 18 830 (47.2%) had a previous episode of sepsis. RS contributed to 3.9% (1667 episodes) and 5.7% (2389 episodes) of community-onset and hospital-onset sepsis episodes, respectively, and RR contributed to 1.8% (796 episodes) and 3.9% (1626 episodes), respectively. Previous infections and exposure to antibiotics were associated with the risk of resistant GNB. For example, in community-onset sepsis, 375 RR episodes (47.1%), 420 RS episodes (25.2%) and 3483 of 40 744 (8.5%) SS episodes were among patients with resistance to antimicrobial drugs (P < .001). The AUROC and AUPRC results varied across hospitals and patient subgroups for both community-onset and hospital-onset sepsis. AUPRC values correlated with the prevalence rates of resistant GNB (R = 0.79; P = .001). In this cohort study of 39 893 patients with sepsis, variable model performance was associated with prevalence rates of antimicrobial resistance rather than patient case mix. This variability suggests caution is needed when using generalized models for predicting resistant GNB etiologies in sepsis.

Performance of Machine Learning Suicide Risk Models in an American Indian Population

Few suicide risk identification tools have been developed specifically for American Indian and Alaska Native populations, even though these populations face the starkest suicide-related inequities. To examine the accuracy of existing machine learning models in a majority American Indian population. This prognostic study used secondary data analysis of electronic health record data collected from January 1, 2017, to December 31, 2021. Existing models from the Mental Health Research Network (MHRN) and Vanderbilt University (VU) were fitted. Models were compared with an augmented screening indicator that included any previous attempt, recent suicidal ideation, or a recent positive suicide risk screen result. The comparison was based on the area under the receiver operating characteristic curve (AUROC). The study was performed in partnership with a tribe and local Indian Health Service (IHS) in the Southwest. All patients were 18 years or older with at least 1 encounter with the IHS unit during the study period. Data were analyzed between October 6, 2022, and July 29, 2024. Suicide attempts or deaths within 90 days. Model performance was compared based on the ability to distinguish between those with a suicide attempt or death within 90 days of their last IHS visit with those without this outcome. Of 16 835 patients (mean [SD] age, 40.0 [17.5] years; 8660 [51.4%] female; 14 251 [84.7%] American Indian), 324 patients (1.9%) had at least 1 suicide attempt, and 37 patients (0.2%) died by suicide. The MHRN model had an AUROC value of 0.81 (95% CI, 0.77-0.85) for 90-day suicide attempts, whereas the VU model had an AUROC value of 0.68 (95% CI, 0.64-0.72), and the augmented screening indicator had an AUROC value of 0.66 (95% CI, 0.63-0.70). Calibration was poor for both models but improved after recalibration. This prognostic study found that existing risk identification models for suicide prevention held promise when applied to new contexts and performed better than relying on a combined indictor of a positive suicide risk screen result, history of attempt, and recent suicidal ideation.

Hepatocellular carcinoma risk scores for non-viral liver disease: A systematic review and meta-analysis

Hepatocellular carcinoma (HCC) risk prediction models may provide a more personalised approach to surveillance for HCC among patients with cirrhosis. This systematic review aims to summarise the performance of HCC prediction models in patients with non-viral chronic liver disease. The study was prospectively registered with PROSPERO (ID: CRD42022370078) and reported in accordance with PRISMA guidelines. MEDLINE and Embase databases were searched using a validated search filter for prediction model studies. Two reviewers independently assessed studies for inclusion and risk of bias. Measures of model performance (discrimination and calibration) to assess the risk of HCC at specified time points were identified. A random effects meta-analysis was performed on a subset of studies that reported performance of the same model. A total of 7,854 studies were identified. After review, 14 studies with a total of 94,014 participants were included; 45% of patients had viral hepatitis, 27% ALD (alcohol-related liver disease) and 19% MASLD (metabolic dysfunction-associated steatotic liver disease). Follow-up ranged from 15.1-138 months. Only one model was developed using a competing risk approach. Age (7 models) and sex (6 models) were the most frequently included predictors. Model discrimination (AUROC or c-statistic) ranged from 0.61-0.947. Only the 'aMAP' score (age, male sex, albumin, bilirubin, and platelets) had sufficient external validation for quantitative analysis, with a pooled c-statistic of 0.81 (95% CI 0.80-0.83). Calibration was reported in only 9 of 14 studies. All studies were rated at high risk of bias. Studies describing risk prediction of HCC in non-viral chronic liver disease are poorly reported, have a high risk of bias and do not account for competing risk events. Patients with ALD and MASLD are underrepresented in development and validation cohorts. These factors remain barriers to the clinical utility and uptake of HCC risk models for those with the most common liver diseases. The recent EASL policy statement emphasises the potential of risk-based surveillance to reduce both hepatocellular carcinoma (HCC)-related deaths and surveillance costs. This study addresses the gap in understanding the performance of current HCC risk models in patients with non-viral liver diseases, reflecting the epidemiological landscape of liver disease in Western countries. In our review of these models we identified several key concerns regarding reporting standards and risk of bias and confirmed that patients with alcohol-related liver disease and metabolic dysfunction-associated steatotic liver disease are underrepresented in model development and validation cohorts. Additionally, most models fail to account for the significant risk of competing events, leading to potential overestimation of true HCC risk. This study highlights these critical issues that may hinder the implementation of risk models in clinical practice and offers key recommendations for future model development studies.

Outcome prediction following lumbar disc surgery: a longitudinal study of outcome trajectories, prognostic factors, and risk models.

This study aimed to 1) describe the 2-year postoperative trajectories of leg pain and overall clinical outcome after surgery for radiculopathy, 2) identify the preoperative prognostic factors that predict trajectories representing poor clinical outcomes, and 3) develop and internally validate multivariable prognostic models to assist with clinical decision-making. This retrospective cohort study included patients enrolled in the Canadian Spine Outcomes and Research Network who were diagnosed with lumbar disc pathology and radiculopathy and had undergone lumbar discectomy at one of 18 spine centers. Potential outcome predictors included preoperative demographic, health-related, and clinical prognostic factors. Clinical outcomes were 1) 2-year univariable latent trajectories of leg pain intensity (numeric pain rating scale) and 2) overall outcomes comprising multivariable trajectories showing the combined postoperative courses of leg and back pain intensity (numeric pain rating scale) together with pain-related disability (Oswestry Disability Index). Each outcome model identified a subgroup of patients classified as experiencing a poor outcome based on minimal change in their clinical status after surgery. Multivariable risk model performance and internal validity were evaluated with discrimination and calibration statistics based on bootstrap shrinkage with 500 resamplings. The authors included data from 1142 patients (47.6% female). The trajectory models identified 3 subgroups based on the patients' postoperative courses of pain or disability: 88.6% of patients in the leg pain model and 71.9% in the overall outcome model experienced a good-to-excellent outcome. The models classified 11.4% (leg pain outcome) and 28.2% (overall outcome) of patients as experiencing a poor clinical outcome, which was defined as minimal improvement in pain or disability after surgery. Eleven individual demographic, health, and clinical factors predicted patients' poor leg pain and overall outcomes. The performance of the multivariable risk model for leg pain was inadequate, while the overall outcome model had acceptable discrimination, calibration, and internal validity for predicting a poor surgical outcome. Patients with lumbar radiculopathy experience heterogeneous postoperative trajectories of pain and disability after lumbar discectomy. Individual preoperative factors are associated with postoperative outcomes and can be combined within a multivariable risk model to predict overall patient outcome. These results may inform clinical practice but require external validation before confident clinical implementation.

A 2-stage model of heterogenous treatment effects for brain atrophy in multiple sclerosis utilizing the MS PATHS research network

Two-stage models of heterogenous treatment effects (HTE) may advance personalized medicine in multiple sclerosis (MS). Brain atrophy is a relatively objective outcome measure that has strong relationships to MS prognosis and treatment effects and is enabled by standardized MRI. To predict brain atrophy outcomes for patients initiating disease-modifying therapies (DMT) with different efficacies, considering the patients' baseline brain atrophy risk measured via brain parenchymal fraction (BPF). Analyses included patients enrolled in the Multiple Sclerosis Partners Advancing Technology and Health Solutions (MS PATHS) network who started DMT and had complete baseline data and ≥ 6-month brain MRI follow-up. All brain MRIs were acquired using standardized acquisition sequences on Siemens 3T scanners. BPF change risk was derived by linear mixed effects models using baseline covariates. Model performance was assessed by predicted versus actual BPF change R2. Propensity score (PS) weighting was used to balance covariates between groups defined by DMT efficacy (high: natalizumab, alemtuzumab, ocrelizumab, and rituximab; moderate: dimethyl fumarate, fingolimod, and cladribine; low: teriflunomide, interferons, and glatiramer acetate). HTE models predicting 1 year change in BPF were built using a weighted linear mixed effects model with low-efficacy DMT as the reference. Analyses included 581 high-, 183 moderate-, and 106 low-efficacy DMT-treated patients. The mean and median number of brain MRI observations per treatment period were 2.9 and 3.0, respectively. Risk model performance R2=0.55. After PS weighting, covariate standardized mean differences were <10 %, indicating excellent balance across measured variables. Changes in BPF between baseline and follow-up were found to be statistically significant (p < 0.001), suggesting a pathological change. Patients with low brain atrophy risk had a similar outcome regardless of DMT selection. In patients with high brain atrophy risk, high- and moderate-efficacy DMTs performed similarly, while a 2-fold worse BPF change was predicted for patients selecting low-efficacy DMTs (p < 0.001). Similar results were observed in a sensitivity analysis adjusting for pseudoatrophy effects in a sub-population of patients treated with natalizumab. The relative benefit of selecting higher efficacy treatments may vary depending on patients' baseline brain atrophy risk. Poor outcomes are predicted in individuals with high baseline risk who are treated with low-efficacy DMTs.

Association of R-loop binding proteins with prognosis and anti-tumor drug sensitivity in lung adenocarcinoma: a bioinfor-matic study.

To investigate the association of R-loop binding proteins with prognosis and chemotherapy efficacy in lung adenocarcinoma. The data related to R-loop regulatory genes were obtained from literature of R-loop proteomics and relevant databases. We used 403 cases of lung adenocarcinoma in the Cancer Genome Atlas as training set, and two datasets GSE14814 and GSE31210 in Gene Expression Omnibus as validation sets. The weighted gene co-expression network analysis (WGCNA) was employed to identify R-loop genes with a significant impact on the clinical phenotype of lung adenocarcinoma. Least absolute shrinkage and selection operator (LASSO) regression analysis was utilized to eliminate genes exhibiting multicollinearity. A multivariate Cox regression analysis was employed to scrutinize clinical variables and R-loop characteristic genes that exert independent prognostic effects on patient survival. Subsequently, a risk score model was constructed. The predictive capacity of this model for the prognosis of patients was analyzed and validated. Additionally, the performance of risk model on the anti-tumor drug sensitivity was assessed. The mutations of R-loop genes were analyzed by maftools. The effect of PLEC expression on anti-tumor drug sensitivity was tested on non-small cell lung adenocarcinoma H1299 and A549 cells in vitro. A collection of 1551 R-loop genes were obtained, and 78 genes exhibited significant effects on the clinical phenotype shown on WGCNA. The LASSO regression analysis retained fourteen R-loop genes. A multivariate Cox regression analysis further identified three R-loop genes (HEXIM1, GLI2, PLEC) and a clinical variable (tumor grading) that were associated with patient prognosis. Risk prediction model was established according to the regression coefficients of each parameter. Kaplan-Meier survival analysis showed that the prognosis of high-risk group was significantly worse than that of low-risk group (P<0.01). The time-dependent ROC curve showed that the risk model had good predictive ability in both training and validation sets. Predictive analyses of anti-neoplastic drug sensitivity indicated a diminished responsiveness to both chemotherapy and targeted treatment drugs among high-risk patients. The expression of PLEC was strongly correlated with sensitivity to gefitinib, a classical EGFR inhibitor. R-loop binding proteins have been identified as significant determinants in the prognosis and therapeutic strategies for lung adenocarcinoma, which indicates that therapeutic interventions targeting these specific R-loop binding proteins might contribute to a better survival of the patients.

Ten-Year Risk Equations for Incident Heart Failure in Established Atherosclerotic Cardiovascular Disease Populations.

Ten-year risk equations for incident heart failure (HF) are available for the general population, but not for patients with established atherosclerotic cardiovascular disease (ASCVD), which is highly prevalent in HF cohorts. This study aimed to develop and validate 10-year risk equations for incident HF in patients with known ASCVD. Ten-year risk equations for incident HF were developed using the United Kingdom Biobank cohort (recruitment 2006-2010) including participants with established ASCVD but free from HF at baseline. Model performance was validated using the Australian Baker Heart and Diabetes Institute Biobank cohort (recruitment 2000-2011) and compared with the performance of general population risk models. Incident HF occurred in 13.7% of the development cohort (n=31 446, median 63 years, 35% women, follow-up 10.7±2.7 years) and in 21.3% of the validation cohort (n=1659, median age 65 years, 25% women, follow-up 9.4±3.7 years). Predictors of HF included in the sex-specific models were age, body mass index, systolic blood pressure (treated or untreated), glucose (treated or untreated), cholesterol, smoking status, QRS duration, kidney disease, myocardial infarction, and atrial fibrillation. ASCVD-HF equations had good discrimination and calibration in development and validation cohorts, with superior performance to general population risk equations. ASCVD-specific 10-year risk equations for HF outperform general population risk models in individuals with established ASCVD. The ASCVD-HF equations can be calculated from readily available clinical data and could facilitate screening and preventative treatment decisions in this high-risk group.

Incident heart failure in chronic kidney disease: proteomics informs biology and risk stratification.

Incident heart failure (HF) among individuals with chronic kidney disease (CKD) incurs hospitalizations that burden patients and health care systems. There are few preventative therapies, and the Pooled Cohort equations to Prevent Heart Failure (PCP-HF) perform poorly in the setting of CKD. New drug targets and better risk stratification are urgently needed. In this analysis of incident HF, SomaScan V4.0 (4638 proteins) was analysed in 2906 participants of the Chronic Renal Insufficiency Cohort (CRIC) with validation in the Atherosclerosis Risk in Communities (ARIC) study. The primary outcome was 14-year incident HF (390 events); secondary outcomes included 4-year HF (183 events), HF with reduced ejection fraction (137 events), and HF with preserved ejection fraction (165 events). Mendelian randomization and Gene Ontology were applied to examine causality and pathways. The performance of novel multi-protein risk models was compared to the PCP-HF risk score. Over 200 proteins were associated with incident HF after adjustment for estimated glomerular filtration rate at P < 1 × 10-5. After adjustment for covariates including N-terminal pro-B-type natriuretic peptide, 17 proteins remained associated at P < 1 × 10-5. Mendelian randomization associations were found for six proteins, of which four are druggable targets: FCG2B, IGFBP3, CAH6, and ASGR1. For the primary outcome, the C-statistic (95% confidence interval [CI]) for the 48-protein model in CRIC was 0.790 (0.735, 0.844) vs. 0.703 (0.644, 0.762) for the PCP-HF model (P = .001). C-statistic (95% CI) for the protein model in ARIC was 0.747 (0.707, 0.787). Large-scale proteomics reveal novel circulating protein biomarkers and potential mediators of HF in CKD. Proteomic risk models improve upon the PCP-HF risk score in this population.

Screening mitochondria-related biomarkers in skin and plasma of atopic dermatitis patients by bioinformatics analysis and machine learning.

There is a significant imbalance of mitochondrial activity and oxidative stress (OS) status in patients with atopic dermatitis (AD). This study aims to screen skin and peripheral mitochondria-related biomarkers, providing insights into the underlying mechanisms of mitochondrial dysfunction in AD. Public data were obtained from MitoCarta 3.0 and GEO database. We screened mitochondria-related differentially expressed genes (MitoDEGs) using R language and then performed GO and KEGG pathway analysis on MitoDEGs. PPI and machine learning algorithms were also used to select hub MitoDEGs. Meanwhile, the expression of hub MitoDEGs in clinical samples were verified. Using ROC curve analysis, the diagnostic performance of risk model constructed from these hub MitoDEGs was evaluated in the training and validation sets. Further computer-aided algorithm analyses included gene set enrichment analysis (GSEA), immune infiltration and mitochondrial metabolism, centered on these hub MitoDEGs. We also used real-time PCR and Spearman method to evaluate the relationship between plasma circulating cell-free mitochondrial DNA (ccf-mtDNA) levels and disease severity in AD patients. MitoDEGs in AD were significantly enriched in pathways involved in mitochondrial respiration, mitochondrial metabolism, and mitochondrial membrane transport. Four hub genes (BAX, IDH3A, MRPS6, and GPT2) were selected to take part in the creation of a novel mitochondrial-based risk model for AD prediction. The risk score demonstrated excellent diagnostic performance in both the training cohort (AUC = 1.000) and the validation cohort (AUC = 0.810). Four hub MitoDEGs were also clearly associated with the innate immune cells' infiltration and the molecular modifications of mitochondrial hypermetabolism in AD. We further discovered that AD patients had considerably greater plasma ccf-mtDNA levels than controls (U = 92.0, p< 0.001). Besides, there was a significant relationship between the up-regulation of plasma mtDNA and the severity of AD symptoms. The study highlights BAX, IDH3A, MRPS6 and GPT2 as crucial MitoDEGs and demonstrates their efficiency in identifying AD. Moderate to severe AD is associated with increased markers of mitochondrial damage and cellular stress (ccf=mtDNA). Our study provides data support for the variation in mitochondria-related functional characteristics of AD patients.

Progression from Prediabetes to Diabetes in a Diverse U.S. Population: A Machine Learning Model.

Objective: To date, there are no widely implemented machine learning (ML) models that predict progression from prediabetes to diabetes. Addressing this knowledge gap would aid in identifying at-risk patients within this heterogeneous population who may benefit from targeted treatment and management in order to preserve glucose metabolism and prevent adverse outcomes. The objective of this study was to utilize readily available laboratory data to train and test the performance of ML-based predictive risk models for progression from prediabetes to diabetes. Methods: The study population was composed of laboratory information services data procured from a large U.S. outpatient laboratory network. The retrospective dataset was composed of 15,029 adults over a 5-year period with initial hemoglobin A1C (A1C) values between 5.0% and 6.4%. ML models were developed using random forest survival methods. The ground truth outcome was progression to A1C values indicative of diabetes (i.e., ≥6.5%) within 5 years. Results: The prediabetes risk classifier model accurately predicted A1C ≥6.5% within 5 years and achieved an area under the receiver-operator characteristic curve of 0.87. The most important predictors of progression from prediabetes to diabetes were initial A1C, initial serum glucose, A1C slope, serum glucose slope, initial HDL, HDL slope, age, and sex. Conclusions: Leveraging readily obtainable laboratory data, our ML risk classifier accurately predicts elevation in A1C associated with progression from prediabetes to diabetes. Although prospective studies are warranted, the results support the clinical utility of the model to improve timely recognition, risk stratification, and optimal management for patients with prediabetes.

Incidence, severity, and risk factors for white spot lesions in adolescent patients treated with clear aligners.

This study was aimed to clarify the incidence, severity, and clinical risk factors for white spot lesions (WSLs) in adolescent patients treated with clear aligners. Pre-treatment and post-treatment intraoral photographs of 203 adolescent patients undergoing clear aligner therapy were retrospectively evaluated to assess the occurrence and severity of WSLs. Information on patients' general oral condition and orthodontic treatment was collected from clinical medical documents, retrospective questionnaires, and ClinCheck® software. Independent risk factors and model performance were determined by multivariate logistic regression and receiver operating characteristic curve analysis. Thirty-five percent of adolescent patients developed WSLs during clear aligner treatment. Logistic regression analysis revealed that the presence of WSLs before treatment (OR 2.484, 95% CI 1.245-4.957), frequency of drinking carbonated beverages (OR 1.508, 95% CI 1.045-2.177), and number of anterior attachments (OR 2.192, 95% CI 1.502-3.198) were risk factors for the occurrence of WSLs in adolescent patients treated with clear aligners (P < .05), whereas the number of times they brushed each day (OR 0.656, 95% CI 0.454-0.947) and frequency of aligner cleaning after eating while wearing them (OR 0.611, 95% CI 0.433-0.861) were protective factors against WSLs (P < .05). The incidence of WSLs was high in adolescent patients treated with clear aligners. Few brushings each day, pre-treatment WSLs, a high frequency of drinking carbonated beverages, a low frequency of aligner cleaning after eating while wearing them, and a high number of anterior attachments are strongly associated with the development of WSLs in adolescent patients treated with clear aligners.

Estimated pulse wave velocity and risk of new-onset heart failure.

As a potential surrogate of carotid-femoral pulse wave velocity, estimated pulse wave velocity (ePWV) has been confirmed to independently predict the cardiovascular events, but the association between ePWV and heart failure has not been well confirmed. Therefore, we performed this cohort study to evaluate the association between ePWV and risk of new-onset heart failure. A total of 98269 employees (mean age: 51.77±12.56years, male accounted for 79.9%) without prior heart failure who participated in the 2006-2007 health examination were selected as the observation cohort, with an average follow-up of 13.85±1.40years. Area under the receiver operator characteristic curve (AUC) of ePWV was calculated in prediction of heart failure. The adjusted Cox proportional hazard models were used to estimate hazard ratios and 95% confidence intervals. The category-free net reclassification index (NRI) was calculated to evaluate the reclassification performance of cardiovascular risk models after adding ePWV. The AUC of ePWV was 0.74 in prediction of heart failure. After adjusting for the traditional cardiovascular risk factors except for age and blood pressure, the risk of new-onset heart failure increased by 35% [hazard ratio (HR): 1.35, 95% confidence interval (CI): 1.33-1.37] for each 1m/s increase in ePWV. Subgroup analysis showed that ePWV was significantly associated with incident heart failure regardless of THE presence (HR: 1.33, 95% CI: 1.31-1.36, P<0.01) or absence (HR: 1.59, 95% CI: 1.46-1.73, P<0.01) of cardiovascular risk factors, male (HR: 1.33, 95% CI: 1.31-1.36, P<0.01) or female (HR: 1.44, 95% CI: 1.38-1.51, P<0.01), young and middle-aged (<52years) (HR: 1.50, 95% CI: 1.41-1.58, P<0.01), or middle-aged and elderly (≥52years) (HR: 1.23, 95% CI: 1.21-1.26, P<0.01). The addition of ePWV to the traditional cardiovascular risk model including age and mean arterial pressure could significantly improve the reclassification ability by 31.1% (category-free NRI=0.311, P<0.01). ePWV was an independent predictor for new-onset heart failure.

Performance of risk models to predict mortality risk for patients with heart failure: evaluation in an integrated health system.

Referral of patients with heart failure (HF) who are at high mortality risk for specialist evaluation is recommended. Yet, most tools for identifying such patients are difficult to implement in electronic health record (EHR) systems. To assess the performance and ease of implementation of Machine learning Assessment of RisK and EaRly mortality in Heart Failure (MARKER-HF), a machine-learning model that uses structured data that is readily available in the EHR, and compare it with two commonly used risk scores: the Seattle Heart Failure Model (SHFM) and Meta-Analysis Global Group in Chronic (MAGGIC) Heart Failure Risk Score. Retrospective, cohort study. Data from 6764 adults with HF were abstracted from EHRs at a large integrated health system from 1/1/10 to 12/31/19. One-year survival from time of first cardiology or primary care visit was estimated using MARKER-HF, SHFM, and MAGGIC. Discrimination was measured by the area under the receiver operating curve (AUC). Calibration was assessed graphically. Compared to MARKER-HF, both SHFM and MAGGIC required a considerably larger amount of data engineering and imputation to generate risk score estimates. MARKER-HF, SHFM, and MAGGIC exhibited similar discriminations with AUCs of 0.70 (0.69-0.73), 0.71 (0.69-0.72), and 0.71 (95% CI 0.70-0.73), respectively. All three scores showed good calibration across the full risk spectrum. These findings suggest that MARKER-HF, which uses readily available clinical and lab measurements in the EHR and required less imputation and data engineering than SHFM and MAGGIC, is an easier tool to identify high-risk patients in ambulatory clinics who could benefit from referral to a HF specialist.

Obstructive Sleep Apnea and Nocturnal Hypoxemia Increase the Cardiovascular Risk in Chilean

To determine the association between the respiratory disturbance index (RDI) and the Time under 90% of SpO2 (CT90%) with CV risk using the predictive model of Framingham 2008. In addition, we analyzed the diagnostic performance of the baseline CV risk model, adjusted for RDI, CT90%, and their combination to predict CV mortality. in patients with clinical suspicion of AOS. Single-center prospective cohort study, including 1560 subjects. All patients underwent a clinical evaluation for OSA, blood pressure, and anthropometric variables. To determine the association of the indices of interest with CV risk, a linear multivariate regression was performed between the RDI or CT90% score. All analyses were performed using R software (R-project), and a p-value < 0.05 was considered statistically significant. RDI and CT90% showed significant differences for CV (p-value= <0.001). In addition, a percentage increase was demonstrated in each quartile of the RDI and CT90% (p-value= <0.001). The RDI and CT90% showed a significant and incremental association with the CV risk of the cohort. However, predictive analyses of CV mortality using the RDI and CT90% were not significant.

Preparing for the bedside-optimizing a postpartum depression risk prediction model for clinical implementation in a health system.

We developed and externally validated a machine-learning model to predict postpartum depression (PPD) using data from electronic health records (EHRs). Effort is under way to implement the PPD prediction model within the EHR system for clinical decision support. We describe the pre-implementation evaluation process that considered model performance, fairness, and clinical appropriateness. We used EHR data from an academic medical center (AMC) and a clinical research network database from 2014 to 2020 to evaluate the predictive performance and net benefit of the PPD risk model. We used area under the curve and sensitivity as predictive performance and conducted a decision curve analysis. In assessing model fairness, we employed metrics such as disparate impact, equal opportunity, and predictive parity with the White race being the privileged value. The model was also reviewed by multidisciplinary experts for clinical appropriateness. Lastly, we debiased the model by comparing 5 different debiasing approaches of fairness through blindness and reweighing. We determined the classification threshold through a performance evaluation that prioritized sensitivity and decision curve analysis. The baseline PPD model exhibited some unfairness in the AMC data but had a fair performance in the clinical research network data. We revised the model by fairness through blindness, a debiasing approach that yielded the best overall performance and fairness, while considering clinical appropriateness suggested by the expert reviewers. The findings emphasize the need for a thorough evaluation of intervention-specific models, considering predictive performance, fairness, and appropriateness before clinical implementation.

Prognostic risk models for incident hypertension: A PRISMA systematic review and meta-analysis.

Our goal was to review the available literature on prognostic risk prediction for incident hypertension, synthesize performance, and provide suggestions for future work on the topic. A systematic search on PUBMED and Web of Science databases was conducted for studies on prognostic risk prediction models for incident hypertension in generally healthy individuals. Study-quality was assessed using the Prediction model Risk of Bias Assessment Tool (PROBAST) checklist. Three-level meta-analyses were used to obtain pooled AUC/C-statistic estimates. Heterogeneity was explored using study and cohort characteristics in meta-regressions. From 5090 hits, we found 53 eligible studies, and included 47 in meta-analyses. Only four studies were assessed to have results with low risk of bias. Few models had been externally validated, with only the Framingham risk model validated more than thrice. The pooled AUC/C-statistics were 0.82 (0.77-0.86) for machine learning models and 0.78 (0.76-0.80) for traditional models, with high heterogeneity in both groups (I2 > 99%). Intra-class correlations within studies were 60% and 90%, respectively. Follow-up time (P = 0.0405) was significant for ML models and age (P = 0.0271) for traditional models in explaining heterogeneity. Validations of the Framingham risk model had high heterogeneity (I2 > 99%). Overall, the quality of included studies was assessed as poor. AUC/C-statistic were mostly acceptable or good, and higher for ML models than traditional models. High heterogeneity implies large variability in the performance of new risk models. Further, large heterogeneity in validations of the Framingham risk model indicate variability in model performance on new populations. To enable researchers to assess hypertension risk models, we encourage adherence to existing guidelines for reporting and developing risk models, specifically reporting appropriate performance measures. Further, we recommend a stronger focus on validation of models by considering reasonable baseline models and performing external validations of existing models. Hence, developed risk models must be made available for external researchers.

Forecasting value-at-risk and expected shortfall in emerging market: does forecast combination help?

PurposeThis paper investigates how various strategies for combining forecasts, both simple and optimised approaches, are compared with popular individual risk models in estimating value-at-risk (VaR) and expected shortfall (ES) in emerging market at alternative risk levels.Design/methodology/approachUsing the case study of the Vietnamese stock market, the author produced one-day-ahead VaR and ES forecast from seven individual risk models and ten alternative forecast combinations. Next, the author employed a battery of backtesting procedures and alternative loss functions to evaluate the global predictive accuracy of the different methods. Finally, the author investigated the relative performance over time of VaR and ES forecasts using fluctuation test.FindingsThe empirical results indicate that, although combined forecasts have reasonable predictive abilities, they are often outperformed by one individual risk model. Furthermore, the author showed that the complex combining methods with optimised weighting functions do not perform better than simple combining methods. The fluctuation test suggests that the poor performance of combined forecasts is mainly due to their inability to cope with periods of instability.Research limitations/implicationsThis study reveals the limitation of combining strategies in the one-day-ahead VaR and ES forecasts in emerging markets. A possible direction for further research is to investigate whether this finding holds for multi-day ahead forecasts. Moreover, the inferior performance of combined forecasts during periods of instability motivates further research on the combining strategies that take into account for potential structure breaks in the performance of individual risk models. A potential approach is to improve the individual risk models with macroeconomic variables using a mixed-data sampling approach.Originality/valueFirst, the authors contribute to the literature on the forecasting combinations for VaR and ES measures. Second, the author explored a wide range of alternative risk models to forecast both VaR and ES with recent data including periods of the COVID-19 pandemic. Although forecast combination strategies have been providing several good results in several fields, the literature of forecast combination in the VaR and ES context is surprisingly limited, especially for emerging market returns. To the best of the author’s knowledge, this is the first study investigating predictive power of combining methods for VaR and ES in an emerging market.

Evaluating and Comparing Risk Model Performance

Evaluating and Comparing Risk Model Performance

Meta-analysis of the performances of contemporary risk models in cardiac surgery for infective endocarditis

Abstract Background Risk scores play important roles in the decision-making for cardiac surgery, but remain under-studied in the setting of infective endocarditis (IE) when there is often a high risk of adverse outcomes. Several IE-specific risk scores have been developed recently, but there is a paucity of external validation studies. We aimed to pool the performance of current risk models applied to IE surgery. Methods Ovid Medline, Ovid Embase and Cochrane Library were comprehensively searched following PRISMA guidelines for eligible studies. Standard meta-analysis techniques using random effects models were used to pool the discriminative and calibration performance of risk scores for predicting operative mortality after IE surgery. Results Amongst 620 studies and 50 full-text articles screened, 17 studies reporting 12 risk scores and totaling 6337 patients were included. Amongst traditional risk scores, EuroSCORE I, EuroSCORE II, and STS IE score had pooled areas under curve (95%CI) of 0.727 (0.700.754), 0.753 (0.731-0.775), and 0.695 (0.645-0.746) (Table1). However, EuroSCORE II significantly over-estimated operative mortality while EuroSCORE I had good calibration with pooled Peto’s odds ratio (95%CI) of 2.47 (1.22-5.01) and 1.00 (0.74-1.36) respectively. Amongst IE-specific scores, areas under curve (95%CI) was highest for the ANCLA score 0.838 (0.803-0.873) and lowest for the COSTA score 0.641 (0.573-0.710). Conclusion Most contemporary risk score had moderate discriminative ability for operative mortality after IE surgery, with the IE-specific ANCLA score performing the best. However, there is room for improving the performance of current risk models, along with need for further external validation studies and calibration assessment, especially for IE-specific scores.