Clinical Risk Prediction Models Research Articles

Using prediction models to improve care and communicate risk: updated modeling for children with IgA nephropathy

Clinical risk prediction models are being generated at an increasing rate. One important component is the identification of groups for whom such models might require recalibration to retain their desired performance. To this end, an update of the postbiopsy International IgA Nephropathy Prediction Tool for children has been published in this issue of Kidney International. We review the methods used and generalizability in practice, along with broader concepts in model development and application.

A cohort study examining predictors of stroke and bleeding among people with congenital heart disease and atrial fibrillation

Abstract Background Congenital Heart Disease (CHD) is associated with development of atrial fibrillation (AF) at an early age. With increased survival of this cohort into adulthood, the prevalence of concurrent CHD and AF is expected to increase over the next several decades. Little is known about the utility of clinical risk factors or prediction models in determining bleeding or stroke rates in adults with CHD and AF. Purpose This study aims to measure the performance of the CHADSVASC and HASBLED scores in a cohort of people with CHD. The association between traditional stroke and bleeding risk factors, as well as CHD-specific factors, will be quantified. Methods Patients age 18 or older attending one of three centers in the USA were included in this study if they had a history of AF on ECG, holter monitor or device interrogation, in addition to a diagnosis of Tetralogy of Fallot (TOF), Double Outlet Right Ventricle (DORV), or Pulmonary Atresia with Intact Ventricular Septum (PA-IVS). Outcomes were major bleeding event, and stroke rate. Analysis was conducted using cox univariable regression controlling for current age. Receiver operator curve and c-statistics were produced for the CHADSVASC and HASBLED scores respectively. Results 326 people were included in this study, of whom 161 (49.5%) had TOF secondary to Pulmonic Stenosis, 126 (38.8%) had TOF secondary to Pulmonary Atresia, 25 (7.7%) had DORV, and 155 (47.7%) had PA-IVS. 227 people used warfarin (79.4%), 28 (9.8%) used apixaban, 9 (3.1%) used Rivaroxaban, 3 (1%) used Edoxaban, and 5 (1.7%) used Enoxaparin for a period of at least 30 days during the study. 13 (4.5%) people didn’t receive anticoagulation during the study. 26 people had a stroke (8.8%) while 85 people (28.6%) had a major bleeding event. Only the CHADSVASC score was significantly associated with stroke rates (HR 2.103, 95% CI 1.555 to 2.844, p<0.001). Clinical variables, such as subtype of CHD, prior conduit placement or unifocalization were not associated with stroke or major bleeding events. C-statistic for the CHADSVASC score in predicting stroke events was 0.755 (Figure 1). C-statistic for the HASBLED score in predicting major bleeding was 0.442 (Figure 2). Conclusions Among people with CHD and AF, the CHADSVASC score can robustly predict stroke events. Other clinical variables are less valuable in predicting either bleeding or strokes. The HASBLED score likely doesn’t have a role in predicting major bleeding episodes in this cohort.

The CTOX-Score, a clinical risk prediction tool for cardiovascular toxicity secondary to cancer therapies

Abstract Introduction Cardiovascular toxicity (cardiotoxicity) is recognized as a major side-effect of cancer therapies. As early diagnosis and management is crucial in clinical practice1,2, knowledge about risk factors and risk scores is scarce3,4. Purpose The aim of this study is to develop a clinical prediction score to identify patients at high/very high risk of cardiotoxicity. Methods The CARDIOTOX registry5,6 included patients receiving cancer therapies related to moderate/high risk for cardiotoxicity (2012-2017). Clinical, laboratory and echo parameters were collected at baseline and at 3 weeks, 3 and 6 months, 1, 1.5 and 2 years after initiation of cancer therapy. Patients were classified according to current cardioncology guidelines2 and divided in two groups: no or mild cardiotoxicity vs moderate or high cardiotoxicity. Difference between groups were tested with t-student or U Mann-Withney for continuous variables and Fisher’s exact test for categorical data. Variables with p<0.05 were included in a multivariate binary logistic regression model to find independent predictors of cardiotoxicity and construct a clinical risk score, CTOX-Score. The calibration was evaluated by Hosmer-Lemeshow goodness-of-fit test and discrimination using ROC curves. Results From the 1324 individuals included in the registry we discarded all those with pre-existing symptomatic heart failure or LVEF<40% at baseline and those with incomplete baseline data. We considered 998 patients, aged 54.62±14.15 years, 176 men (17.6%), of which 53 (5.3%) developed moderate or severe cardiotoxicity. These patients had lower baseline LVEF (59.91±7.50% vs 64.20±5.88%), higher creatinine levels (0.90 (0.76, 1.10) mg/dl vs 0.74 (0.62, 0.90) mg/dl) and higher NT-ProBNP (177 (58, 476) pg/ml vs 66 (38, 133) pg/ml) compared with patients with no or mild cardiotoxicity (p<0.001). We found previous radiotherapy (OR=4.66, CI:1.24-17.52), acute myeloid leukemia (OR=6.85; CI:1.16-40.50), monoclonal antibodies (OR=3.27; CI:1.35-7.89), LVEF (OR=0.96; CI:0.84-0.98) and serum creatinine (OR=5.43; CI:1.63-18.14) as independent predictors of cardiotoxicity (p<0.05). A clinical risk score was developed based in beta coefficient of variables (Table 1). The model has good calibration according to Hosmer-Lemeshow goodness-of-fit (χ2=2.800, p=0.424) and good discrimination power (AUC:0.780, CI:0.69-0.87; p<0.001). The best cut-off value of CTOX-Score is ≥3 points (Sensitivity:70.00%, Specificity:72.25%, NPV:98.11%). Patients were classified in four risk categories: low (0-1 points); moderate (2); high (3-4); very high (≥5) (Figure 1). The model has good accuracy to identify very high-risk patients (Specificity 98.80%). Conclusion The CTOX-Score is a clinical risk prediction model to identify patients at high/very high-risk of cardiotoxicity and could allow for more personalized monitoring of patients. Additional studies are needed to externally validate the model prior to implementation in clinical practice.Multivariate binary logistic regressionCTOX-Score values among population

Development and validation of a risk prediction model for worsening renal function in patients with incident heart failure

Abstract Background Worsening renal function (WRF) is one of the strongest predictors of outcome in patients with heart failure (HF). The presence of WRF often influences the decision to start, up-titrate, or discontinue disease modifying HF therapies and is one of the key determinants of suboptimal guideline-directed medical therapy. Existing WRF risk scores were developed in hospitalised HF patients to predict in-hospital WRF. Their performance among incident HF is far from ideal. It is therefore important to develop a new WRF risk prediction model in people with newly diagnosed HF. Purpose This study was to develop and validate a clinical risk prediction model for 180-day WRF post diagnosis of incident HF using data from a population-based longitudinal cohort. Methods We developed and validated a multivariable logistic regression model for WRF in the 180 days post diagnosis of incident HF. Data for individuals with incident HF between 2016 to 2021 were extracted from the NELSON study conducted in Tayside, Scotland. WRF was defined as a composite event of (i) having two consecutive eGFRs declined by 40% or greater; (ii) having an eGFR < 15mL/min/1.73m2; (iii) initiation of sustained dialysis; (iv) development of end-stage kidney disease; (v) receiving kidney transplantation; or (vi) kidney related death. Four candidate models included one model using literature knowledge; one multivariable fractional polynomial (MFP) model; and two stepwise selected models based on either Akaike or Bayesian information criterion (AIC or BIC). Calibration and discrimination were assessed, and performance stability were evaluated using optimism corrected performance via 1000 times bootstrapping. Results 4076 individuals (mean age 72.8 ± 13.2; 58% male; 1081 HF with reduced ejection fraction (HFrEF), 646 with HF with mildly reduced ejection fraction (HFmrEF), 1348 HF with preserved ejection fraction (HFpEF), and 1001 HF with unknown ejection fraction (EF)) who were WRF-free when diagnosed with HF were identified. Of these, 2095 (51.4%) were in-patients, and 1981 (48.6%) were out-patients. 285 (7%) patients developed WRF within 180 days post HF diagnosis. The selected MFP model (comprising 12 predictors, Table 1) showed good calibration (slope = 1.00 [0.86, 1.14]; intercept = 0.0 [-0.13, 0.13]) discrimination (C-statistic = 0.69 [95% CI: 0.65, 0.72]), and stability (optimism corrected: calibration slope = 0.94 [0.80, 1.07]; calibration intercept = -0.14 [-0.50, 0.20]; C-statistic = 0.67 [0.64, 0.71]). Discrimination of the developed model significantly outperformed two existing risk scores (Forman and Basel, both with a C-statistic = 0.62 [0.59 – 0.65]). Conclusions We have shown that the developed model performed better in predicting WRF in individuals with newly diagnosed HF. This could have utility in identifying at risk patients with HF early in the disease trajectory for therapies that can reduce renal decline.Calibration of predicted riskModel estimates for predicting 180d WRF

Clinical utility of an artificial intelligence radiomics-based tool for risk stratification of pulmonary nodules.

Clinical utility data on pulmonary nodule (PN) risk stratification biomarkers are lacking. We aimed to determine the incremental predictive value and clinical utility of using an artificial intelligence (AI) radiomics-based computer-aided diagnosis (CAD) tool in addition to routine clinical information to risk stratify PNs among real-world patients. We performed a retrospective cohort study of patients with PNs who underwent lung biopsy. We collected clinical data and used a commercially available AI radiomics-based CAD tool to calculate a Lung Cancer Prediction (LCP) score. We developed logistic regression models to evaluate a well-validated clinical risk prediction model (the Mayo Clinic model) with and without the LCP score (Mayo vs Mayo + LCP) using area under the curve (AUC), risk stratification table, and standardized net benefit analyses. Among the 134 patients undergoing PN biopsy, cancer prevalence was 61%. Addition of the radiomics-based LCP score to the Mayo model was associated with increased predictive accuracy (likelihood ratio test, P = .012). The AUCs for the Mayo and Mayo + LCP models were 0.58 (95% CI = 0.48 to 0.69) and 0.65 (95% CI = 0.56 to 0.75), respectively. At the 65% risk threshold, the Mayo + LCP model was associated with increased sensitivity (56% vs 38%; P = .019), similar false positive rate (33% vs 35%; P = .8), and increased standardized net benefit (18% vs -3.3%) compared with the Mayo model. Use of a commercially available AI radiomics-based CAD tool as a supplement to clinical information improved PN cancer risk prediction and may result in clinically meaningful changes in risk stratification.

Abstract P102: Reducing Testing Burden in Patients Screened for Primary Aldosteronism

Introduction: Primary aldosteronism (PA) is an increasingly recognized secondary cause of hypertension characterized by higher cardiovascular, renal, and metabolic disease rates than blood pressure-matched essential hypertensives. Current literature suggests expanding screening for PA to include the majority, if not all, hypertensive patients. This expansive screening would place considerable burden on patients and the healthcare system. A simple clinical risk prediction model that identifies patients with low risk for PA could reduce this testing burden. Methods: We evaluated 111 hypertensive patients in the Southern Illinois University Hypertension Clinic who underwent a screening test and, if positive, had subsequent confirmatory testing for PA. We characterized patients based on routine clinical measures including antihypertensive treatment intensity, blood pressure, serum electrolytes, and body size. Classification trees incorporating 96 patients without missing data points for any considered predictor variable were used to identify hypertensive screen-positive patients who have low to absent risk for PA as determined by confirmatory testing. Results: Patients with PA had higher serum aldosterone levels at screening (mean = 22.5 ng/dl ± SD = 11.6 vs. 15.2 ± 4.5, p < 0.001), higher BMI (35.8 kg/m 2 ± 9.9 vs. 29.9 ± 5.8, p < 0.001), lower potassium (3.9 mmol/L ± 0.5 vs. 4.1 ± 0.6, p = 0.03), and more intense antihypertensive drug therapy (defined by the World Health Organization defined daily dose [6.1 ± 3.2 vs. 4.4 ± 3.0, p = 0.01]) than hypertensive patients without PA. The optimal classification tree model performed relatively poorly at differentiating those with PA from those without PA (sensitivity = 0.76, specificity = 0.86, positive predictive value = 0.87, and negative predictive value = 0.73). However, the model did reveal a subset of individuals who consistently tested negative for PA. Out of 14 patients with serum aldosterone concentration < 25.5, serum potassium > 3.55, and BMI < 27, none had PA. Conclusion: We identified a set of readily available clinical variables for identifying hypertensive patients who screened positive for PA but in whom PA was subsequently proven to be absent. Applying these criteria to our screen-positive hypertensive cohort reduces confirmatory testing by approximately 15%. Study in a larger sample size and external validation of this model is needed prior to recommending it for use in clinical decision-making.

Risk factors for the development of refeeding syndrome in adults: A systematic review.

Identifying patients with a particularly high risk of refeeding syndrome (RFS) is essential for taking preventive measures. To guide the development of clinical decision-making and risk prediction models or other screening tools for RFS, increased knowledge of risk factors is needed. Therefore, we conducted a systematic review to identify risk factors for the development of RFS. PubMed, EMBASE, Cochrane Library, and Web of Science were searched from January 1990 until March 2023. Studies investigating demographic, clinical, drug use, laboratory, and/or nutrition factors for RFS were considered. The Newcastle-Ottawa Scale was used to appraise the methodological quality of included studies. Of 1589 identified records, 30 studies were included. Thirty-three factors associated with increased risk of RFS after multivariable adjustments were identified. The following factors were reported by two or more studies, with 0-1 study reporting null findings: a previous history of alcohol misuse, cancer, comorbid hypertension, high Acute Physiology and Chronic Health Evaluation II score, high Sequential Organ Failure Assessment score, low Glasgow coma scale score, the use of diuretics before refeeding, low baseline serum prealbumin level, high baseline level of creatinine, and enteral nutrition. The majority of the studies (20, 66.7%) were of high methodological quality. In conclusion, this systematic review informs on several risk factors for RFS in patients. To improve risk stratification and guide development of risk prediction models or other screening tools, further confirmation is needed because there were a small number of studies and a low number of high-quality studies on each factor.

Multiple biomarkers are equivalent to clinical pulmonary arterial hypertension survival risk models

Risk assessment in pulmonary arterial hypertension (PAH) is fundamental to guiding treatment and improved outcomes. Clinical models are excellent at identifying high-risk patients but leave uncertainty amongst moderate risk patients. Can a multiple blood biomarker model of PAH, using previously described biomarkers, improve risk discrimination over current models? Using multiplex ELISA, we measured NT-proBNP, ST2, IL-6, Endostatin, Galectin-3, HDGF, and IGF binding proteins (IGFBP1-7) in train (n=1623), test (n=696) and validation (n=237) cohorts. Clinical variables, biomarkers were evaluated by principal component analysis. NT-proBNP was not included to develop an NT-proBNP independent model. Unsupervised k-means clustering classified subjects into clusters. Transplant-free survival by cluster was examined using Kaplan-Meier and Cox proportional hazard regressions. Hazard by cluster was compared to NT-proBNP, REVEAL, and ESC/ERS Risk models alone, and combined clinical and biomarker models. The algorithm generated 5 clusters with good risk discrimination using 6 biomarkers, weight, height, and age at PAH diagnosis. In the test and validation cohorts the biomarker model alone performed equivalent to REVEAL (AUC 0.74). Adding the biomarker model to the ESC/ERS, and REVEAL scores improved the ESC/ERS and REVEAL scores. The best overall model was the biomarker model adjusted for NT-proBNP with the best C-statistic, AIC, and calibration for the adjusted model compared to either the biomarker or NT-proBNP model alone. A multi-biomarker model alone was equivalent to current PAH clinical mortality risk prediction models and improved performance when combined, and added to NT-proBNP. Clinical risk scores offer excellent predictive models but require multiple tests; adding blood biomarkers to models can improve prediction or enable more frequent, non-invasive monitoring of risk in PAH to support therapeutic decision making.

Association between growth differentiation factor-15 and adverse outcomes among patients with heart failure: A systematic literature review

Growth differentiation factor-15 (GDF-15) is an emerging biomarker in several conditions. This SLR, conducted following PRISMA guidelines, examined the association between GDF-15 concentration and range of adverse outcomes in patients with heart failure (HF). Publications were identified from Embase® and Medline® bibliographic databases between January 1, 2014, and August 23, 2022 (congress abstracts: January 1, 2020, to August 23, 2022). Sixty-three publications met the eligibility criteria (55 manuscripts and 8 abstracts; 45 observational studies and 18 post hoc analyses of randomized controlled trials [RCTs]). Of the 19 outcomes identified, the most frequently reported longitudinal outcomes were mortality (n = 32 studies; all-cause [n = 27] or cardiovascular-related [n = 6]), composite outcomes (n = 28; most commonly mortality ± hospitalization/rehospitalization [n = 19]), and hospitalization/re-hospitalization (n = 11). The most common cross-sectional outcome was renal function (n = 22). Among longitudinal studies assessing independent relationships with outcomes using multivariate analyses (MVA), a significant increase in risk associated with higher baseline GDF-15 concentration was found in 22/24 (92 %) studies assessing all-cause mortality, 4/5 (80 %) assessing cardiovascular-related mortality, 13/19 (68 %) assessing composite outcomes, and 4/8 (50 %) assessing hospitalization/rehospitalization. All (7/7; 100 %) of the cross-sectional studies assessing the relationship with renal function by MVA, and 3/4 (75 %) assessing exercise capacity, found poorer outcomes associated with higher baseline GDF-15 concentrations. This SLR suggests GDF-15 is an independent predictor of mortality and other adverse but nonfatal outcomes in patients with HF. A better understanding of the prognostic role of GDF-15 in HF could improve clinical risk prediction models and potentially help optimize treatment regimens.

Integration of genetic and clinical risk factors improves the risk classification of uveitis in patients with juvenile idiopathic arthritis.

Juvenile idiopathic arthritis (JIA)-associated uveitis (JIAU) is a serious JIA comorbidity that can result in vision impairment. This study aimed to identify genetic risk factors, within the major histocompatibility complex , for JIAU and evaluate their contribution for improving risk classification when combined with clinical risk factors. Data on single nucleotide polymorphisms, amino acids and classical human leukocyte antigen (HLA) alleles were available for 2,497 JIA patients without uveitis and 579 JIAU patients (female=2060, male=1015). Analysis was restricted to patients with inferred European ancestry. Forward conditional logistic regression identified genetic markers exceeding a Bonferroni corrected significance (6x10-6). Multivariable logistic regression estimated the effects of clinical and genetic risk factors and a likelihood ratio test calculated the improvement in model fit when adding genetic factors. Uveitis risk classification performance of a model integrating genetic and clinical risk factors was estimated using area under the receiver operator characteristic curve and compared to a model of clinical risk factors alone. Three genetic risk factors were identified mapping to HLA-DRB1, HLA-DPB1 and HLA-A. These markers were statistically independent from clinical risk factors and significantly improved the fit of a model when included with clinical risk factors (P = 3.3x10-23). The addition of genetic markers improved the classification of JIAU compared to a model of clinical risk factors alone (AUC 0.75 vs. 0.71). Integration of a genetic and clinical risk prediction model outperforms a model based solely on clinical risk factors. Future JIAU risk prediction models should include genetic risk factors.

Predictability of buprenorphine-naloxone treatment retention: A multi-site analysis combining electronic health records and machine learning.

Opioid use disorder (OUD) and opioid dependence lead to significant morbidity and mortality, yet treatment retention, crucial for the effectiveness of medications like buprenorphine-naloxone, remains unpredictable. Our objective was to determine the predictability of 6-month retention in buprenorphine-naloxone treatment using electronic health record (EHR) data from diverse clinical settings and to identify key predictors. This retrospective observational study developed and validated machine learning-based clinical risk prediction models using EHR data. Data were sourced from Stanford University's healthcare system and Holmusk's NeuroBlu database, reflecting a wide range of healthcare settings. The study analyzed 1800 Stanford and 7957 NeuroBlu treatment encounters from 2008 to 2023 and from 2003 to 2023, respectively. Predict continuous prescription of buprenorphine-naloxone for at least 6months, without a gap of more than 30 days. The performance of machine learning prediction models was assessed by area under receiver operating characteristic (ROC-AUC) analysis as well as precision, recall and calibration. To further validate our approach's clinical applicability, we conducted two secondary analyses: a time-to-event analysis on a single site to estimate the duration of buprenorphine-naloxone treatment continuity evaluated by the C-index and a comparative evaluation against predictions made by three human clinical experts. Attrition rates at 6months were 58% (NeuroBlu) and 61% (Stanford). Prediction models trained and internally validated on NeuroBlu data achieved ROC-AUCs up to 75.8 (95% confidence interval [CI] = 73.6-78.0). Addiction medicine specialists' predictions show a ROC-AUC of 67.8 (95% CI = 50.4-85.2). Time-to-event analysis on Stanford data indicated a median treatment retention time of 65 days, with random survival forest model achieving an average C-index of 65.9. The top predictor of treatment retention identified included the diagnosis of opioid dependence. US patients with opioid use disorder or opioid dependence treated with buprenorphine-naloxone prescriptions appear to have a high (∼60%) treatment attrition by 6months. Machine learning models trained on diverse electronic health record datasets appear to be able to predict treatment continuity with accuracy comparable to that of clinical experts.

Development and external validation of a head and neck cancer risk prediction model.

Head and neck cancer (HNC) incidence is on the rise, often diagnosed at late stage and associated with poor prognoses. Risk prediction tools have a potential role in prevention and early detection. The IARC-ARCAGE European case-control study was used as the model development dataset. A clinical HNC risk prediction model using behavioral and demographic predictors was developed via multivariable logistic regression analyses. The model was then externally validated in the UK Biobank cohort. Model performance was tested using discrimination and calibration metrics. 1926 HNC cases and 2043 controls were used for the development of the model. The development dataset model including sociodemographic, smoking, and alcohol variables had moderate discrimination, with an area under curve (AUC) value of 0.75 (95% CI, 0.74-0.77); the calibration slope (0.75) and tests were suggestive of good calibration. 384 616 UK Biobank participants (with 1177 HNC cases) were available for external validation of the model. Upon external validation, the model had an AUC of 0.62 (95% CI, 0.61-0.64). We developed and externally validated a HNC risk prediction model using the ARCAGE and UK Biobank studies, respectively. This model had moderate performance in the development population and acceptable performance in the validation dataset. Demographics and risk behaviors are strong predictors of HNC, and this model may be a helpful tool in primary dental care settings to promote prevention and determine recall intervals for dental examination. Future addition of HPV serology or genetic factors could further enhance individual risk prediction.

Understanding random resampling techniques for class imbalance correction and their consequences on calibration and discrimination of clinical risk prediction models

ObjectiveClass imbalance is sometimes considered a problem when developing clinical prediction models and assessing their performance. To address it, correction strategies involving manipulations of the training dataset, such as random undersampling or oversampling, are frequently used. The aim of this article is to illustrate the consequences of these class imbalance correction strategies on clinical prediction models’ internal validity in terms of calibration and discrimination performances. MethodsWe used both heuristic intuition and formal mathematical reasoning to characterize the relations between conditional probabilities of interest and probabilities targeted when using random undersampling or oversampling. We propose a plug-in estimator that represents a natural correction for predictions obtained from models that have been trained on artificially balanced datasets (“naïve” models). We conducted a Monte Carlo simulation with two different data generation processes and present a real-world example using data from the International Stroke Trial database to empirically demonstrate the consequences of applying random resampling techniques for class imbalance correction on calibration and discrimination (in terms of Area Under the ROC, AUC) for logistic regression and tree-based prediction models. ResultsAcross our simulations and in the real-world example, calibration of the naïve models was very poor. The models using the plug-in estimator generally outperformed the models relying on class imbalance correction in terms of calibration while achieving the same discrimination performance. ConclusionRandom resampling techniques for class imbalance correction do not generally improve discrimination performance (i.e., AUC), and their use is hard to justify when aiming at providing calibrated predictions. Improper use of such class imbalance correction techniques can lead to suboptimal data usage and less valid risk prediction models.

Effect of novel polygenetic variant of HUNT lung cancer model on lung cancer risk assessment over clinical model among light smokers (average <9 pack-years).

e20066 Background: As lung cancer incidence and mortality increase, methods of early diagnosis are needed to increase survival. The validated HUNT Lung Cancer Model (HUNT LCM) predicts individual 6-year risk of lung cancer in ever smokers with an AUC of 0.87 based on eight clinical variables: sex, age, BMI, pack-years, smoking intensity number of cigarettes per day, quit time, daily cough, and daily indoors smoke exposure. We aimed to explore whether the performance of the HUNT model can be improved by adding genetic information, and in what group of ever smokers these differences in performance emerges. Methods: Based on the eight clinical variables of the HUNT LCM and 22 single nucleotide polymorphisms (SNPs) previously identified in the literature as highly associated with lung cancer risk, the novel lung cancer risk prediction model HUNT Lung-SNP was developed. Clinical and genetical data of ever-smokers from the HUNT2 population study (n = 30746, median follow-up 15.26 years) where 160 individuals were diagnosed with lung cancer within six years were used for the model fitting. External validation was performed in another population-based cohort, the Tromsø Study (n = 2663) where 39 were diagnosed with lung cancer within 6 years. Results: The integrated discrimination improvement index (IDI) between the HUNT Lung-SNP and the HUNT LCM shows that the HUNT Lung-SNP significantly improve the LC risk stratification within six years with an IDI of 0.019 (95% CI 0.015-0.025), p < 0·001 and of 0.013 (95% CI 0.008-0.018), p < 0·001 in the HUNT2 and Tromsø cohorts, respectively. Of the cases, 21/160 (13%) and 10/39 (25%) in the HUNT and Tromsø cohort respectively, were low risk by the HUNT LCM but high risk by the HUNT Lung-SNP model. The main difference in characteristics among cases predicted by the HUNT Lung-SNP model, but missed by the HUNT LCM, was the average number of pack-years: 8.22 vs. 29.8 (p < 0.001) in HUNT2 and 7.7 vs. 33.7 (p < 0.001) in Tromsø cohort. Conclusions: A genetic approach combined with clinical variables of the HUNT LCM can improve the lung cancer risk prediction significantly by increasing the sensitivity among light smokers. Thus, we identified a high-risk population in two independent cohorts that would not be eligible for screening by clinical cutoffs nor with a clinical risk prediction model. This novel HUNT Lung-SNP needs further validation in other populations to evaluate its clinical utility.

ViTR-SP: A CT-based vision transformer model for prediction of pneumonitis in patients with non-small cell lung cancer who received thoracic radiotherapy and immunotherapy.

e20034 Background: The combination of thoracic radiotherapy (TRT) and immunotherapy accompanied by the accumulation of pulmonary toxicity. This study aimed to construct a deep radiomics model based on sequential computed tomography (CT) images for radioimmune-associated pneumonitis prediction in non-small cell lung cancer (NSCLC) patients treated with TRT and immune checkpoint inhibitors (ICIs). Methods: Three models were constructed based on a vision transformer (ViT) network using sequential chest CT images, tumor-mask and lung-mask as inputs, respectively. NSCLC patients treated with TRT and ICIs at Shandong Cancer Hospital and Qilu Hospital were included for model training and external validation, respectively. The image-level model was trained based on ViT to identify predictive images, and the image-level probability distribution were performed. Then, a ViT-Recursive Neural Network (ViT-RNN) model framework was constructed at the patient-level to integrate the image-level information to derive the final prediction, which was evaluated in an independent validation cohort. Results: A total of 230 NSCLC patients from Shandong Cancer Hospital were randomly assigned in a 7:3 ratio for model training and validation, respectively. Another 42 patients from Qilu Hospital were used as an independent external validation cohort. The ViT-RNN model for symptomatic pneumonitis (ViTR-SP) based on sequential CT images achieved better area under curve (AUC), sensitivity, and specificity of 0.725, 0.750, and 0.886 than lung-mask based model (0.575, 0.625, and 0.705) and tumor-mask based model (0.658, 0.667, and 0.773) in the internal validation set. In the external validation set, the proposed model achieved the AUC, sensitivity, and specificity of 0.654, 0.818, 0.727, while the lung-mask and tumor-mask based models had the AUC, sensitivity, and specificity of 0.540, 0.545, 0.677, and 0.630, 0.545, 0.742, respectively. The performance of the model was improved after integrating clinical features, with internal and external validation AUC of 0.782 and 0.672, respectively. Conclusions: In conclusion, the clinical combined ViTR-SP neural network risk prediction model based on sequential chest CT images can be used for predicting the risk of developing radioimmune-associated pneumonitis in NSCLC patients.

Predicting the risk of pancreatic cancer in adults with new-onset diabetes: development and internal–external validation of a clinical risk prediction model

BackgroundThe National Institute for Health and Care Excellence (NICE) recommends that people aged 60+ years with newly diagnosed diabetes and weight loss undergo abdominal imaging to assess for pancreatic cancer. More nuanced stratification could lead to enrichment of these referral pathways.MethodsPopulation-based cohort study of adults aged 30–85 years at type 2 diabetes diagnosis (2010–2021) using the QResearch primary care database in England linked to secondary care data, the national cancer registry and mortality registers. Clinical prediction models were developed to estimate risks of pancreatic cancer diagnosis within 2 years and evaluated using internal–external cross-validation.ResultsSeven hundred and sixty-seven of 253,766 individuals were diagnosed with pancreatic cancer within 2 years. Models included age, sex, BMI, prior venous thromboembolism, digoxin prescription, HbA1c, ALT, creatinine, haemoglobin, platelet count; and the presence of abdominal pain, weight loss, jaundice, heartburn, indigestion or nausea (previous 6 months). The Cox model had the highest discrimination (Harrell’s C-index 0.802 (95% CI: 0.797–0.817)), the highest clinical utility, and was well calibrated. The model’s highest 1% of predicted risks captured 12.51% of pancreatic cancer cases. NICE guidance had 3.95% sensitivity.DiscussionA new prediction model could have clinical utility in identifying individuals with recent onset diabetes suitable for fast-track abdominal imaging.

Benchmarking clinical risk prediction algorithms with ensemble machine learning for the noninvasive diagnosis of liver fibrosis in NAFLD.

Ensemble machine-learning methods, like the superlearner, combine multiple models into a single one to enhance predictive accuracy. Here we explore the potential of the superlearner as a benchmarking tool for clinical risk prediction, illustrating the approach to identifying significant liver fibrosis among patients with NAFLD. We used 23 demographic/clinical variables to train superlearner(s) on data from the NASH-clinical research network observational study (n = 648) and validated models with data from the FLINT trial (n = 270) and National Health and Nutrition Examination Survey (NHANES) participants with NAFLD (n = 1244). Comparing the superlearner's performance to existing models (Fibrosis-4 [FIB-4], NAFLD fibrosis score, Forns, AST to Platelet Ratio Index [APRI], BARD, and Steatosis-Associated Fibrosis Estimator [SAFE]), it exhibited strong discriminative ability in the FLINT and NHANES validation sets, with AUCs of 0.79 (95% CI: 0.73-0.84) and 0.74 (95% CI: 0.68-0.79) respectively. Notably, the SAFE score performed similarly to the superlearner, both of which outperformed FIB-4, APRI, Forns, and BARD scores in the validation data sets. Surprisingly, the superlearner derived from 12 base models matched the performance of one with 90 base models. Overall, the superlearner, being the "best-in-class" machine-learning predictor, excelled in detecting fibrotic NASH, and this approach can be used to benchmark the performance of conventional clinical risk prediction models.

Development and validation of a practical clinical risk prediction model for post-endoscopic retrograde cholangiopancreatography pancreatitis.

Pancreatitis following endoscopic retrograde cholangiopancreatography (ERCP) can lead to significant morbidity and mortality. We aimed to develop an accurate post-ERCP pancreatitis risk prediction model using easily obtainable variables. Using prospective multi-center ERCP data, we performed logistic regression using stepwise selection on several patient-, procedure-, and endoscopist-related factors that were determined a priori. The final model was based on a combination of the Bayesian information criterion and Akaike's information criterion performance, balancing the inclusion of clinically relevant variables and model parsimony. All available data were used for model development, with subsequent internal validation performed on bootstrapped data using 10-fold cross-validation. Data from 3021 ERCPs were used to inform models. There were 151 cases of post-ERCP pancreatitis (5.0% incidence). Variables included in the final model included female sex, pancreatic duct cannulation, native papilla status, pre-cut sphincterotomy, increasing cannulation time, presence of biliary stricture, patient age, and placement of a pancreatic duct stent. The final model was discriminating, with a receiver operating characteristic curve statistic of 0.79, and well-calibrated, with a predicted risk-to-observed risk ratio of 1.003. We successfully developed and internally validated a promising post-ERCP pancreatitis clinical prediction model using easily obtainable variables that are known at baseline or observed during the ERCP procedure. The model achieved an area under the curve of 0.79. External validation is planned as additional data becomes available.

Clinical risk prediction model and external validation of positive surgical margin in laparoscopic radical prostatectomy based on MRI lesion location.

To clarify the composition of lesions in different magnetic resonance imaging (MRI) partitions of positive surgical margins (PSM) after laparoscopic radical prostatectomy, explore the influence of lesion location on PSM, and construct a clinical prediction model to predict the risk of PSM. This retrospective cohort study included 309 patients who underwent laparoscopic radical prostatectomy from 2018 to 2021 in our center was performed. 129 patients who met the same criteria from January to September 2022 were external validation cohorts. The incidence of PSM in transition zone (TZ) lesions was higher than that in peripheral zone (PZ) lesions. The incidence of PSM in the middle PZ was lower than that in other regions. Prostate specific antigen (PSA), clinical T-stage, the number of positive cores, international society of urological pathology (ISUP) grade (biopsy), MRI lesion location, extracapsular extension, seminal vesicle invasion (SVI), pseudo-capsule invasion (PCI), long diameter of lesions, lesion volume, lesion volume ratio, PSA density were related to PSM. MRI lesion location and PCI were independent risk factors for PSM. Least absolute shrinkage and selection operator (LASSO) regression was used to construct a clinical prediction model for PSM, including five variables: the number of positive cores, SVI, MRI lesion location, long diameter of lesions, and PSA. The positive rate of surgical margin in middle PZ was significantly lower than that in other regions, and MRI lesion location was an independent risk factor for PSM.

Predicting Risk of Heroin Overdose, Remission, Use, and Mortality Using Ensemble Learning Methods in a Cohort of People with Heroin Dependence

AbstractDespite decades of research demonstrating the effectiveness of treatments for heroin dependence, rates of heroin use, dependence, and death have dramatically increased over the past decade. While evidence has highlighted a range of risk and protective factors for relapse, remission, and other outcomes, this presents clinicians with the challenge as to how to synthesise and integrate the evolving evidence-base to guide clinical decision-making and facilitate the provision of personalised healthcare. Using data from the 11-year follow-up of the Australian Treatment Outcome Study (ATOS), we aimed to develop a clinical risk prediction model to assist clinicians calculate the risk of a range of heroin-related outcomes at varying follow-up intervals for their clients based on known risk factors. Between 2001 and 2002, 615 people with heroin dependence were recruited as part of a prospective longitudinal cohort study. An ensemble machine learning approach was applied to predict risk of heroin use, remission, overdose, and mortality at 1-, 5-, and 10 + year post-study entry. Variables most consistently ranked in the top 10 in terms of their level of importance across outcomes included age; age first got high, used heroin, or injected; sexual trauma; years of school completed; prison history; severe mental health disability; past month criminal involvement; and past month benzodiazepine use. This study provides clinically relevant information on key risk factors associated with heroin use, remission, non-fatal overdose, and mortality among people with heroin dependence, to help guide clinical decision-making in the selection and tailoring of interventions to ensure that the ‘right treatment’ is delivered to the ‘right person’ at the ‘right time’.