Validation Of Risk Prediction Model Research Articles

International External Validation of Risk Prediction Model of 90-Day Mortality after Gastrectomy for Cancer Using Machine Learning

Background: Radical gastrectomy remains the main treatment for gastric cancer, despite its high mortality. A clinical predictive model of 90-day mortality (90DM) risk after gastric cancer surgery based on the Spanish EURECCA registry database was developed using a matching learning algorithm. We performed an external validation of this model based on data from an international multicenter cohort of patients. Methods: A cohort of patients from the European GASTRODATA database was selected. Demographic, clinical, and treatment variables in the original and validation cohorts were compared. The performance of the model was evaluated using the area under the curve (AUC) for a random forest model. Results: The validation cohort included 2546 patients from 24 European hospitals. The advanced clinical T- and N-category, neoadjuvant therapy, open procedures, total gastrectomy rates, and mean volume of the centers were significantly higher in the validation cohort. The 90DM rate was also higher in the validation cohort (5.6%) vs. the original cohort (3.7%). The AUC in the validation model was 0.716. Conclusion: The externally validated model for predicting the 90DM risk in gastric cancer patients undergoing gastrectomy with curative intent continues to be as useful as the original model in clinical practice.

Development and validation of risk prediction model for post-donation renal function in living kidney donors

This study aimed to create and validate a predictive model for renal function following live kidney donation, using pre-donation factors. Accurately predicting remaining renal function post live kidney donation is currently insufficient, necessitating an effective assessment tool. A multicenter retrospective study of 2318 live kidney donors from two independent centers (May 2007–December 2019) was conducted. The primary endpoint was the reduction in eGFR to below 60 mL/min/m2 6 months post-donation. The primary endpoint was achieved in 14.4% of the training cohort and 25.8% of the validation cohort. Sex, age, BMI, hypertension, preoperative eGFR, and remnant kidney proportion (RKP) measured by computerized tomography (CT) volumetry were found significant in the univariable analysis. These variables informed a scoring system based on multivariable analysis: sex (male: 1, female: 0), age at operation (< 30: 0, 30–39: 1, 40–59: 2, ≥ 60: 3), preoperative eGFR (≥ 100: 0, 90–99: 2, 80–89: 4, < 80: 5), and RKP (≥ 52%: 0, < 52%: 1). The total score ranged from 0 to 10. The model showed good discrimination for the primary endpoint in both cohorts. The prediction model provides a useful tool for estimating post-donation renal dysfunction risk, factoring in the side of the donated kidney. It offers potential enhancement to pre-donation evaluations.

International External Validation of Risk Prediction Model of 90-Day Mortality after Gastrectomy for Cancer Using Machine Learning.

Radical gastrectomy remains the main treatment for gastric cancer, despite its high mortality. A clinical predictive model of 90-day mortality (90DM) risk after gastric cancer surgery based on the Spanish EURECCA registry database was developed using a matching learning algorithm. We performed an external validation of this model based on data from an international multicenter cohort of patients. A cohort of patients from the European GASTRODATA database was selected. Demographic, clinical, and treatment variables in the original and validation cohorts were compared. The performance of the model was evaluated using the area under the curve (AUC) for a random forest model. The validation cohort included 2546 patients from 24 European hospitals. The advanced clinical T- and N-category, neoadjuvant therapy, open procedures, total gastrectomy rates, and mean volume of the centers were significantly higher in the validation cohort. The 90DM rate was also higher in the validation cohort (5.6%) vs. the original cohort (3.7%). The AUC in the validation model was 0.716. The externally validated model for predicting the 90DM risk in gastric cancer patients undergoing gastrectomy with curative intent continues to be as useful as the original model in clinical practice.

Risk of emergency hospital admission related to adverse events after antibiotic treatment in adults with a common infection: impact of COVID-19 and derivation and validation of risk prediction models

BackgroundWith the global challenge of antimicrobial resistance intensified during the COVID-19 pandemic, evaluating adverse events (AEs) post-antibiotic treatment for common infections is crucial. This study aims to examines the changes in incidence rates of AEs during the COVID-19 pandemic and predict AE risk following antibiotic prescriptions for common infections, considering their previous antibiotic exposure and other long-term clinical conditions.MethodsWith the approval of NHS England, we used OpenSAFELY platform and analysed electronic health records from patients aged 18–110, prescribed antibiotics for urinary tract infection (UTI), lower respiratory tract infections (LRTI), upper respiratory tract infections (URTI), sinusitis, otitis externa, and otitis media between January 2019 and June 2023. We evaluated the temporal trends in the incidence rate of AEs for each infection, analysing monthly changes over time. The survival probability of emergency AE hospitalisation was estimated in each COVID-19 period (period 1: 1 January 2019 to 25 March 2020, period 2: 26 March 2020 to 8 March 2021, period 3: 9 March 2021 to 30 June 2023) using the Kaplan–Meier approach. Prognostic models, using Cox proportional hazards regression, were developed and validated to predict AE risk within 30 days post-prescription using the records in Period 1.ResultsOut of 9.4 million patients who received antibiotics, 0.6% of UTI, 0.3% of URTI, and 0.5% of LRTI patients experienced AEs. UTI and LRTI patients demonstrated a higher risk of AEs, with a noted increase in AE incidence during the COVID-19 pandemic. Higher comorbidity and recent antibiotic use emerged as significant AE predictors. The developed models exhibited good calibration and discrimination, especially for UTIs and LRTIs, with a C-statistic above 0.70.ConclusionsThe study reveals a variable incidence of AEs post-antibiotic treatment for common infections, with UTI and LRTI patients facing higher risks. AE risks varied between infections and COVID-19 periods. These findings underscore the necessity for cautious antibiotic prescribing and call for further exploration into the intricate dynamics between antibiotic use, AEs, and the pandemic.

Analysis of risk factors and construction of a prediction model for posttraumatic stress disorder among Chinese college students during the COVID-19 pandemic

BackgroundTo explore the risk factors of post-traumatic stress disorder (PTSD) among Chinese college students during the COVID-19 pandemic and the construction and validation of risk prediction models. MethodsA total of 10,705 university students were selected for the study. The questionnaire included the Generalized Anxiety Disorder 7 (GAD-7), Patient Health Questionnaire 9 (PHQ-9), PTSD Checklist for DSM-5 (PCL-5), and self-designed questionnaire. These assessments were conducted to facilitate the survey, construct the predictive model and validate the model's validity. ResultsSex, left-behind experience, poverty status, anxiety score, and depression score were identified as independent risk factors influencing psychological trauma among Chinese college students during the COVID-19 pandemic, while COVID-19 infection emerged as a protective factor against psychological trauma. A column chart was constructed to visualize the six independent risk factors derived from logistic regression analysis. The Hosmer–Lemeshow test results (χ2 = 13.021, P = 0.111) indicated that the risk prediction model fitted well. The receiver operating characteristic (ROC) curve showed an area under the curve (AUC) of 0.864 in the model group and 0.855 in the validation group. The calibration curves of the model closely resembled the ideal curve. Decision curve analysis (DCA) revealed that the model provided net benefit and demonstrated good clinical utility. LimitationsThe validation of the model is currently restricted to internal assessments. However, further confirmation through larger sample sizes, multicenter investigations, and prospective studies is necessary. ConclusionsThe model effectively predicted PTSD risk among Chinese college students during the COVID-19 pandemic, indicating strong clinical applicability.

Construction and validation of risk prediction models for pulmonary embolism in hospitalized patients based on different machine learning methods.

This study aims to apply different machine learning (ML) methods to construct risk prediction models for pulmonary embolism (PE) in hospitalized patients, and to evaluate and compare the predictive efficacy and clinical benefit of each model. We conducted a retrospective study involving 332 participants (172 PE positive cases and 160 PE negative cases) recruited from Guangdong Medical University. Participants were randomly divided into a training group (70%) and a validation group (30%). Baseline data were analyzed using univariate analysis, and potential independent risk factors associated with PE were further identified through univariate and multivariate logistic regression analysis. Six ML models, namely Logistic Regression (LR), Decision Tree (DT), Random Forest (RF), Naive Bayes (NB), Support Vector Machine (SVM), and AdaBoost were developed. The predictive efficacy of each model was compared using the receiver operating characteristic (ROC) curve analysis and the area under the curve (AUC). Clinical benefit was assessed using decision curve analysis (DCA). Logistic regression analysis identified lower extremity deep venous thrombosis, elevated D-dimer, shortened activated partial prothrombin time, and increased red blood cell distribution width as potential independent risk factors for PE. Among the six ML models, the RF model achieved the highest AUC of 0.778. Additionally, DCA consistently indicated that the RF model offered the greatest clinical benefit. This study developed six ML models, with the RF model exhibiting the highest predictive efficacy and clinical benefit in the identification and prediction of PE occurrence in hospitalized patients.

What's New in Dementia Risk Prediction Modelling? An Updated Systematic Review.

Identifying individuals at high risk of dementia is critical to optimized clinical care, formulating effective preventative strategies, and determining eligibility for clinical trials. Since our previous systematic reviews in 2010 and 2015, there has been a surge in dementia risk prediction modelling. The aim of this study was to update our previous reviews to explore, and critically review, new developments in dementia risk modelling. MEDLINE, Embase, Scopus, and Web of Science were searched from March 2014 to June 2022. Studies were included if they were population- or community-based cohorts (including electronic health record data), had developed a model for predicting late-life incident dementia, and included model performance indices such as discrimination, calibration, or external validation. In total, 9,209 articles were identified from the electronic search, of which 74 met the inclusion criteria. We found a substantial increase in the number of new models published from 2014 (>50 new models), including an increase in the number of models developed using machine learning. Over 450 unique predictor (component) variables have been tested. Nineteen studies (26%) undertook external validation of newly developed or existing models, with mixed results. For the first time, models have also been developed in low- and middle-income countries (LMICs) and others validated in racial and ethnic minority groups. The literature on dementia risk prediction modelling is rapidly evolving with new analytical developments and testing in LMICs. However, it is still challenging to make recommendations about which one model is the most suitable for routine use in a clinical setting. There is an urgent need to develop a suitable, robust, validated risk prediction model in the general population that can be widely implemented in clinical practice to improve dementia prevention.

#1584 Development and validation of risk prediction models according to different clinical settings of IgA nephropathy: nationwide multicenter cohort study

Abstract Background and Aims IgA nephropathy (IgAN) is the most common glomerulonephritis with a high risk of progression to end-stage renal disease (ESRD). Because machine learning models developed for research have limited use in clinical settings, this study developed and validated prediction models for IgAN according to different clinical settings. Method In this study, we analyzed data from 1,174 patients in the Japanese Nationwide Retrospective Cohort Study in IgAN, focusing on a 10-year observation period. We defined the derivation data from January 2002 to April 2004, and temporal validation data from May 2004 to December 2004. The composite renal outcome was defined as a 1.5-fold increase in creatinine or progression to ESRD. We developed and evaluated the three prediction models according to the different clinical settings: the primary care model for general physicians in primary care settings, the tertiary care model for specialists in tertiary care hospitals, and the research institute model, a machine learning-based approach for academic research institute settings. Results After excluding 82 patients for missing data, 114 and 14 patients experienced the composite renal outcome in the derivation (n = 874) and validation (n = 218) cohorts, respectively. The primary care model identified three variables—eGFR &amp;lt;45 mL/min/1.73 m², proteinuria ≥0.5 g/day, and non-use of corticosteroids—as significant predictors (C-statistic: 0.796; 95% CI: 0.686-0.895). The tertiary care model composed of three variables including glomerular number, histological severity and tubular/interstitial severity showed a slightly higher C-statistic (0.807; 95% CI: 0.713-0.886). When stratifying patients into risk groups, the primary care model showed a higher incidence of composite renal events in high-risk patients, and the tertiary care model effectively discriminated outcomes in all groups. The research institute model, which included 38 variables including proteinuria level, hypertension, eGFR, and non-use of corticosteroids as key predictors, showed comparable performance (C-statistic: 0.808; 95% CI: 0.689-0.886). Conclusion The primary care and tertiary care models provide simple yet effective predictive tools, competitive with the machine learning-based research institution model. These models are critical for predicting renal outcomes in patients with IgA nephropathy and for patient management according to different clinical settings.

Development and validation of risk prediction model for bacterial infections in acute liver failure patients.

Infections significantly increase mortality in acute liver failure (ALF) patients, and there are no risk prediction models for early diagnosis and treatment of infections in ALF patients. This study aims to develop a risk prediction model for bacterial infections in ALF patients to guide rational antibiotic therapy. The data of ALF patients admitted to the Second Hospital of Hebei Medical University in China from January 2017 to January 2022 were retrospectively analyzed for training and internal validation. Patients were selected according to the updated 2011 American Association for the Study of Liver Diseases position paper on ALF. Serological indicators and model scores were collected within 24 h of admission. New models were developed using the multivariate logistic regression analysis. An optimal model was selected by receiver operating characteristic (ROC) analysis, Hosmer-Lemeshow test, the calibration curve, the Brier score, the bootstrap resampling, and the decision curve analysis. A nomogram was plotted to visualize the results. A total of 125 ALF patients were evaluated and 79 were included in the training set. The neutrophil-to-lymphocyte ratio and sequential organ failure assessment (SOFA) were integrated into the new model as independent predictive factors. The new SOFA-based model outperformed other models with an area under the ROC curve of 0.799 [95% confidence interval (CI): 0.652-0.926], the superior calibration and predictive performance in internal validation. High-risk individuals with a nomogram score ≥26 are recommended for antibiotic therapy. The new SOFA-based model demonstrates high accuracy and clinical utility in guiding antibiotic therapy in ALF patients.

Development and validation of risk prediction model for recurrent cardiovascular events among Chinese: the Personalized CARdiovascular DIsease risk Assessment for Chinese model.

Cardiovascular disease (CVD) is a leading cause of mortality, especially in developing countries. This study aimed to develop and validate a CVD risk prediction model, Personalized CARdiovascular DIsease risk Assessment for Chinese (P-CARDIAC), for recurrent cardiovascular events using machine learning technique. Three cohorts of Chinese patients with established CVD were included if they had used any of the public healthcare services provided by the Hong Kong Hospital Authority (HA) since 2004 and categorized by their geographical locations. The 10-year CVD outcome was a composite of diagnostic or procedure codes with specific International Classification of Diseases, Ninth Revision, Clinical Modification. Multivariate imputation with chained equations and XGBoost were applied for the model development. The comparison with Thrombolysis in Myocardial Infarction Risk Score for Secondary Prevention (TRS-2°P) and Secondary Manifestations of ARTerial disease (SMART2) used the validation cohorts with 1000 bootstrap replicates. A total of 48 799, 119 672 and 140 533 patients were included in the derivation and validation cohorts, respectively. A list of 125 risk variables were used to make predictions on CVD risk, of which 8 classes of CVD-related drugs were considered interactive covariates. Model performance in the derivation cohort showed satisfying discrimination and calibration with a C statistic of 0.69. Internal validation showed good discrimination and calibration performance with C statistic over 0.6. The P-CARDIAC also showed better performance than TRS-2°P and SMART2. Compared with other risk scores, the P-CARDIAC enables to identify unique patterns of Chinese patients with established CVD. We anticipate that the P-CARDIAC can be applied in various settings to prevent recurrent CVD events, thus reducing the related healthcare burden.

Validating Risk Prediction Models for Multiple Primaries and Competing Cancer Outcomes in Families With Li-Fraumeni Syndrome Using Clinically Ascertained Data.

There exists a barrier between developing and disseminating risk prediction models in clinical settings. We hypothesize that this barrier may be lifted by demonstrating the utility of these models using incomplete data that are collected in real clinical sessions, as compared with the commonly used research cohorts that are meticulously collected. Genetic counselors (GCs) collect family history when patients (ie, probands) come to MD Anderson Cancer Center for risk assessment of Li-Fraumeni syndrome, a genetic disorder characterized by deleterious germline mutations in the TP53 gene. Our clinical counseling-based (CCB) cohort consists of 3,297 individuals across 124 families (522 cases of single primary cancer and 125 cases of multiple primary cancers). We applied our software suite LFSPRO to make risk predictions and assessed performance in discrimination using AUC and in calibration using observed/expected (O/E) ratio. For prediction of deleterious TP53 mutations, we achieved an AUC of 0.78 (95% CI, 0.71 to 0.85) and an O/E ratio of 1.66 (95% CI, 1.53 to 1.80). Using the LFSPRO.MPC model to predict the onset of the second cancer, we obtained an AUC of 0.70 (95% CI, 0.58 to 0.82). Using the LFSPRO.CS model to predict the onset of different cancer types as the first primary, we achieved AUCs between 0.70 and 0.83 for sarcoma, breast cancer, or other cancers combined. We describe a study that fills in the critical gap in knowledge for the utility of risk prediction models. Using a CCB cohort, our previously validated models have demonstrated good performance and outperformed the standard clinical criteria. Our study suggests that better risk counseling may be achieved by GCs using these already-developed mathematical models.

Incidence and development of validated mortality prediction model among asphyxiated neonates admitted to neonatal intensive care unit at Felege Hiwot Comprehensive Specialized Hospital, Bahir Dar, Northwest Ethiopia, 2021: retrospective follow-up study

IntroductionPerinatal asphyxia is failure to maintain normal breathing at birth. World Health Organization indicates that perinatal asphyxia is the third major cause of neonatal mortality in developing countries accounting for 23% of neonatal deaths every year. At global and national level efforts have done to reduce neonatal mortality, however fatalities from asphyxia remains high in Ethiopia (24%). And there are no sufficient studies to show incidence and prediction of mortality among asphyxiated neonates. Developing validated risk prediction model is one of the crucial strategies to improve neonatal outcomes with asphyxia. Therefore, this study will help to screen asphyxiated neonate at high-risk for mortality during admission by easily accessible predictors. This study aimed to determine the incidence and develop validated Mortality Prediction model among asphyxiated neonates admitted to the Neonatal Intensive Care Unit at Felege-Hiwot Comprehensive Specialized Hospital, Bahir Dar, Ethiopia.MethodRetrospective follow-up study was conducted at Felege-Hiwot Comprehensive Specialized Hospital from September 1, 2017, to March 31, 2021. Simple random sampling was used to select 774 neonates, and 738 were reviewed. Since was data Secondary, it was collected by checklist. After the description of the data by table and graph, Univariable with p-value < 0.25, and stepwise multivariable analysis with p-value < 0.05 were done to develop final reduced prediction model by likelihood ratio test. To improve clinical utility, we developed a simplified risk score to classify asphyxiated neonates at high or low-risk of mortality. The accuracy of the model was evaluated using area under curve, and calibration plot. To measures all accuracy internal validation using bootstrapping technique were assessed. We evaluated the clinical impact of the model using a decision curve analysis across various threshold probabilities.ResultIncidence of neonatal mortality with asphyxia was 27.2% (95% CI: 24.1, 30.6). Rural residence, bad obstetric history, amniotic fluid status, multiple pregnancy, birth weight (< 2500 g), hypoxic-ischemic encephalopathy (stage II and III), and failure to suck were identified in the final risk prediction score. The area under the curve for mortality using 7 predictors was 0.78 (95% CI 0.74 to 0.82). With ≥ 7 cutoffs the sensitivity and specificity of risk prediction score were 0.64 and 0.82 respectively.Conclusion and recommendationIncidence of neonatal mortality with asphyxia was high. The risk prediction score had good discrimination power built by rural residence, bad obstetric history, stained amniotic fluid, multiple pregnancy, birth weight (< 2500 g), hypoxic-ischemic encephalopathy (stage II and III), and failure to suck. Thus, using this score chart and improve neonatal and maternal service reduce mortality among asphyxiated neonates.

Patient outcomes following medical emergency team review on general wards: Development of predictive models.

To develop and internally validate risk prediction models for subsequent clinical deterioration, unplanned ICU admission and death among ward patients following medical emergency team (MET) review. A retrospective cohort study of 1500 patients who remained on a general ward following MET review at an Australian quaternary hospital. Logistic regression was used to model (1) subsequent MET review within 48 h, (2) unplanned ICU admission within 48 h and (3) hospital mortality. Models included demographic, clinical and illness severity variables. Model performance was evaluated using discrimination and calibration with optimism-corrected bootstrapped estimates. Findings are reported using the TRIPOD guideline for multivariable prediction models for prognosis or diagnosis. There was no patient or public involvement in the development and conduct of this study. Within 48 h of index MET review, 8.3% (n = 125) of patients had a subsequent MET review, 7.2% (n = 108) had an unplanned ICU admission and in-hospital mortality was 16% (n = 240). From clinically preselected predictors, models retained age, sex, comorbidity, resuscitation limitation, acuity-dependency profile, MET activation triggers and whether the patient was within 24 h of hospital admission, ICU discharge or surgery. Models for subsequent MET review, unplanned ICU admission, and death had adequate accuracy in development and bootstrapped validation samples. Patients requiring MET review demonstrate complex clinical characteristics and the majority remain on the ward after review for deterioration. A risk score could be used to identify patients at risk of poor outcomes after MET review and support general ward clinical decision-making. Our risk calculator estimates risk for patient outcomes following MET review using clinical data available at the bedside. Future validation and implementation could support evidence-informed team communication and patient placement decisions.

Development and validation of prediction model for fall accidents among chronic kidney disease in the community.

The population with chronic kidney disease (CKD) has significantly heightened risk of fall accidents. The aim of this study was to develop a validated risk prediction model for fall accidents among CKD in the community. Participants with CKD from the China Health and Retirement Longitudinal Study (CHARLS) were included. The study cohort underwent a random split into a training set and a validation set at a ratio of 70 to 30%. Logistic regression and LASSO regression analyses were applied to screen variables for optimal predictors in the model. A predictive model was then constructed and visually represented in a nomogram. Subsequently, the predictive performance was assessed through ROC curves, calibration curves, and decision curve analysis. A total of 911 participants were included, and the prevalence of fall accidents was 30.0% (242/911). Fall down experience, BMI, mobility, dominant handgrip, and depression were chosen as predictor factors to formulate the predictive model, visually represented in a nomogram. The AUC value of the predictive model was 0.724 (95% CI 0.679-0.769). Calibration curves and DCA indicated that the model exhibited good predictive performance. In this study, we constructed a predictive model to assess the risk of falls among individuals with CKD in the community, demonstrating good predictive capability.

Construction and validation of risk prediction model for gastrointestinal bleeding in patients after coronary artery bypass grafting

This study aimed to develop a risk prediction model for gastrointestinal bleeding in patients after coronary artery bypass grafting (CABG) and assessed its accuracy. A retrospective analysis was conducted on 232 patients who underwent CABG under general anesthesia in our hospital between January 2022 and December 2022. The patients were divided into gastrointestinal bleeding (GIB) group (n = 52) and group without gastrointestinal bleeding (non-GIB) (n = 180). The independent risk factors for gastrointestinal bleeding in post-CABG patients were analyzed using χ2 test, t test and logistic multivariate regression analysis. A prediction model was established based on the identified risk factors. To verify the accuracy of the prediction model, a verification group of 161 patients who met the criteria was selected between January to June 2023, and the Bootstrap method was used for internal validation. The discrimination of the prediction model was evaluated using the area under the curve (AUC), where a higher AUC indicates a stronger discrimination effect of the model. The study developed a risk prediction model for gastrointestinal bleeding after CABG surgery. The model identified four independent risk factors: duration of stay in the intensive care unit (ICU) (OR 0.761), cardiopulmonary bypass time (OR 1.019), prolonged aortic occlusion time (OR 0.981) and re-operation for bleeding (OR 0.180). Based on these factors, an individualized risk prediction model was constructed. The C-index values of the modeling group and the verification group were 0.805 [95% CI (0.7303–0.8793)] and 0.785 [95% CI (0.6932–0.8766)], respectively, which indicated a good accuracy and discrimination of this model. The calibration and standard curves showed similar results, which further supported the accuracy of the risk prediction model. In conclusion, ICU time, cardiopulmonary bypass time, aortic occlusion time and re-operation for bleeding are identified as independent risk factors for gastrointestinal bleeding in patients after CABG. The risk prediction model developed in this study demonstrates strong predictive performance and provides valuable insights for clinical medical professionals in evaluating gastrointestinal complications in CABG patients.

Development and Validation of a Risk Prediction Model to Estimate the Risk of Stroke Among Hypertensive Patients in University of Gondar Comprehensive Specialized Hospital, Gondar, 2012 to 2022.

A risk prediction model to predict the risk of stroke has been developed for hypertensive patients. However, the discriminating power is poor, and the predictors are not easily accessible in low-income countries. Therefore, developing a validated risk prediction model to estimate the risk of stroke could help physicians to choose optimal treatment and precisely estimate the risk of stroke. This study aims to develop and validate a risk prediction model to estimate the risk of stroke among hypertensive patients at the University of Gondar Comprehensive Specialized Hospital. A retrospective follow-up study was conducted among 743 hypertensive patients between September 01/2012 and January 31/2022. The participants were selected using a simple random sampling technique. Model performance was evaluated using discrimination, calibration, and Brier scores. Internal validity and clinical utility were evaluated using bootstrapping and a decision curve analysis. Incidence of stroke was 31.4 per 1000 person-years (95% CI: 26.0, 37.7). Combinations of six predictors were selected for model development (sex, residence, baseline diastolic blood pressure, comorbidity, diabetes, and uncontrolled hypertension). In multivariable logistic regression, the discriminatory power of the model was 0.973 (95% CI: 0.959, 0.987). Calibration plot illustrated an overlap between the probabilities of the predicted and actual observed risks after 10,000 times bootstrap re-sampling, with a sensitivity of 92.79%, specificity 93.51%, and accuracy of 93.41%. The decision curve analysis demonstrated that the net benefit of the model was better than other intervention strategies, starting from the initial point. An internally validated, accurate prediction model was developed and visualized in a nomogram. The model is then changed to an offline mobile web-based application to facilitate clinical applicability. The authors recommend that other researchers eternally validate the model.

Using Multi-Modal Electronic Health Record Data for the Development and Validation of Risk Prediction Models for Long COVID Using the Super Learner Algorithm

Background: Post-Acute Sequelae of COVID-19 (PASC) have emerged as a global public health and healthcare challenge. This study aimed to uncover predictive factors for PASC from multi-modal data to develop a predictive model for PASC diagnoses. Methods: We analyzed electronic health records from 92,301 COVID-19 patients, covering medical phenotypes, medications, and lab results. We used a Super Learner-based prediction approach to identify predictive factors. We integrated the model outputs into individual and composite risk scores and evaluated their predictive performance. Results: Our analysis identified several factors predictive of diagnoses of PASC, including being overweight/obese and the use of HMG CoA reductase inhibitors prior to COVID-19 infection, and respiratory system symptoms during COVID-19 infection. We developed a composite risk score with a moderate discriminatory ability for PASC (covariate-adjusted AUC (95% confidence interval): 0.66 (0.63, 0.69)) by combining the risk scores based on phenotype and medication records. The combined risk score could identify 10% of individuals with a 2.2-fold increased risk for PASC. Conclusions: We identified several factors predictive of diagnoses of PASC and integrated the information into a composite risk score for PASC prediction, which could contribute to the identification of individuals at higher risk for PASC and inform preventive efforts.

Abstract 17853: Efficacy of Grace Score for Prediction of Mortality in Patients of Acute Coronary Syndrome: A Meta-Analysis

Introduction: The Global Registry of Acute Coronary Syndrome (GRACE) is a validated risk prediction model for acute coronary syndromes (ACS). However, discrimination ability of this model for predicting mortality in ACS remains unclear. The aim of this meta-analysis is to determine the predictive accuracy of GRACE score for mortality in ACS patients. Methods: A comprehensive literature search was conducted across MEDLINE, Embase &amp; the Cochrane Library from inception till May 2023. All studies that reported the C-statistic for predicting in-hospital, 6-month, 1 year &amp; 2-year mortality in ACS patients were included. The generic inverse variance weighted random effects method was used to pool C-statistics &amp; corresponding standard errors (SEs) from each study. Subgroup analysis was conducted based on whether the patients required percutaneous coronary intervention (PCI) or not. Results: A total of 22 studies with a patient population of 132856 were included. For in-hospital mortality (n=99774), the GRACE score showed a good discrimination ability to predict mortality (summary C-statistic 0.86 [0.85-0.87], I 2 =58%). The pooled C-statistic for 6-month mortality (n=24174) was 0.80 (0.70-0.84; I 2 =93%). For 1-year mortality (n=13517), the predictive accuracy of the GRACE score was moderate (summary estimate= 0.77 [0.70-0.83]; I 2 =94%). The GRACE score proved to be moderately accurate in predicting 2-year mortality (n=4972) in ACS patients [0.75 (0.66-0.83); I 2 =66%]. All the findings were consistent across the two subgroups of PCI vs no PCI (p for interaction &gt;0.05). Conclusion: The GRACE score is a simple scoring system that demonstrated good predictive ability for in hospital mortality, &amp; at 6-months while moderate predictive ability for the risk of death at 1-year, &amp; 2-year in patients of ACS with &amp; without PCI.

Risk model-based management for second primary lung cancer among lung cancer survivors through a validated risk prediction model.

Recent therapeutic advances and screening technologies have improved survival among patients with lung cancer, who are now at high risk of developing second primary lung cancer (SPLC). Recently, an SPLC risk-prediction model (called SPLC-RAT) was developed and validated using data from population-based epidemiological cohorts and clinical trials, but real-world validation has been lacking. The predictive performance of SPLC-RAT was evaluated in a hospital-based cohort of lung cancer survivors. The authors analyzed data from 8448 ever-smoking patients diagnosed with initial primary lung cancer (IPLC) in 1997-2006 at Mayo Clinic, with each patient followed for SPLC through 2018. The predictive performance of SPLC-RAT and further explored the potential of improving SPLC detection through risk model-based surveillance using SPLC-RAT versus existing clinical surveillance guidelines. Of 8448 IPLC patients, 483 (5.7%) developed SPLC over 26,470 person-years. The application of SPLC-RAT showed high discrimination area under the receiver operating characteristics curve: 0.81). When the cohort was stratified by a 10-year risk threshold of ≥5.6% (i.e., 80th percentile from the SPLC-RAT development cohort), the observed SPLC incidence was significantly elevated in the high-risk versus low-risk subgroup (13.1% vs. 1.1%, p<1×10-6 ). The risk-based surveillance through SPLC-RAT (≥5.6% threshold) outperformed the National Comprehensive Cancer Network guidelines with higher sensitivity (86.4% vs. 79.4%) and specificity (38.9% vs. 30.4%) and required 20% fewer computed tomography follow-ups needed to detect one SPLC (162 vs. 202). In a large, hospital-based cohort, the authors validated the predictive performance of SPLC-RAT in identifying high-risk survivors of SPLC and showed its potential to improve SPLC detection through risk-based surveillance. Lung cancer survivors have a high risk of developing second primary lung cancer (SPLC). However, no evidence-based guidelines for SPLC surveillance are available for lung cancer survivors. Recently, an SPLC risk-prediction model was developed and validated using data from population-based epidemiological cohorts and clinical trials, but real-world validation has been lacking. Using a large, real-world cohort of lung cancer survivors, we showed the high predictive accuracy and risk-stratification ability of the SPLC risk-prediction model. Furthermore, we demonstrated the potential to enhance efficiency in detecting SPLC using risk model-based surveillance strategies compared to the existing consensus-based clinical guidelines, including the National Comprehensive Cancer Network.

Facilitating Harmonization of Variables in Framingham, MESA, ARIC, and REGARDS Studies Through a Metadata Repository.

High-quality research in cardiovascular prevention, as in other fields, requires inclusion of a broad range of data sets from different sources. Integrating and harmonizing different data sources are essential to increase generalizability, sample size, and representation of understudied populations-strengthening the evidence for the scientific questions being addressed. Here, we describe an effort to build an open-access repository and interactive online portal for researchers to access the metadata and code harmonizing data from 4 well-known cohort studies-the REGARDS (Reasons for Geographic and Racial Differences in Stroke) study, FHS (Framingham Heart Study), MESA (Multi-Ethnic Study of Atherosclerosis), and ARIC (Atherosclerosis Risk in Communities) study. We introduce a methodology and a framework used for preprocessing and harmonizing variables from multiple studies. We provide a real-case study and step-by-step guidance to demonstrate the practical utility of our repository and interactive web page. In addition to our successful development of such an open-access repository and interactive web page, this exercise in harmonizing data from multiple cohort studies has revealed several key themes. These themes include the importance of careful preprocessing and harmonization of variables, the value of creating an open-access repository to facilitate collaboration and reproducibility, and the potential for using harmonized data to address important scientific questions and disparities in cardiovascular disease research. By integrating and harmonizing these large-scale cohort studies, such a repository may improve the statistical power and representation of understudied cohorts, enabling development and validation of risk prediction models, identification and investigation of risk factors, and creating a platform for racial disparities research. URL: https://precision.heart.org/duke-ninds.