External Validation Set Research Articles

A practical nomogram and risk stratification system for predicting in-hospital mortality of patients with non-acute myocardial infarction related cardiogenic shock

Abstract Background The morbidity and mortality of Cardiogenic Shock(CS) remain high despite the advance in management these years1.As acute myocardial infarction (AMI) is considered to be the most common reason for cardiogenic shock,Non-AMI related CS is excluded from the majority of CS studies and remains largely understudied2. Considering its inherent complexity and hemodynamically multiplicity, it is needed to focusing on these patients to identify specific risk factors, which may be conducive to the promotion of critically CS patients with early decision and timely intervention. Purpose In this study,we sought to investigate the variables which are predictive of in hospital mortality in non AMI related CS patients, developed and validated a practical nomogram for risk quantification. Methods 1298 patients and 548 patients with cardiogenic shock from the MIMIC-IV (version 1.0) and MIMIC-III(version1.4) databaes were included in the study after excluding acute myocardial infarction. We used in-hospital death and 30 day survival as the end points. Lasson and logistic regression analysis were used to identify statistically significant predictors which were finally involved in the nomogram. The predictive performance of the nomogram was evaluated by Harrell’s concordance index (C-index) and calibration after comparing with commonly used ICU scores for in-hospital death.Lastly, we plotted Kaplan-Meier curves of three subgroup classified by the identified cutoff values of the nomogram to further assess the discriminative ability of the prognosis. Results The in-hospital mortality of patients with non-AMI related CS is about 36.6%.After the analysis of regression, age, heart rate, wbc, albumin, lactic acid, GCS score, urine volume, using vasopressor were identified as the most critical factors for the in-hospital death. Based on the these results, a nomogram for predicting in-hospital death was established.The calibration plot also shows good consistency between nomogram prediction and actual in-hospital mortality.In addition, compared with other scores, the AUC value of nomogram was 0.806 (95% CI = 0.799-0.831) in the training queue, 0.814 (95% CI = 0.771-0.852) in the internal validation queue and 0.730 (95% CI = 0.690-0.767) in the external validation set, which was significantly higher than other commonly used ICU scoring systems(SAPSII, APSIII and SOFA).Also, three risk subgroups were created using two boundary points (303 and 339) of the nomogram score and the survival curve shows that there were significant differences in the 30 day survival of each subgroup. Conclusion A prognostic nomogram and risk stratification system of non-AMI related CS were established and verified, which can provide a practical tool for individualized clinical management of patients.Workflow of the study.Nomogram for non-AMI related CS patients

Multiomics Profiling of Plasma Reveals Molecular Alterations Prior to a Diagnosis with Stroke Among Chinese Hypertension Patients.

We aimed to investigate the correlation between plasma proteins and metabolites and the occurrence of future strokes using mass spectrometry and bioinformatics as well as to identify other biomarkers that could predict stroke risk in hypertensive patients. In a nested case-control study, baseline plasma samples were collected from 50 hypertensive subjects who developed stroke and 50 gender-, age- and body mass index-matched controls. Plasma untargeted metabolomics and data independent acquisition-based proteomics analysis were performed in hypertensive patients, and 19 metabolites and 111 proteins were found to be differentially expressed. Integrative analyses revealed that molecular changes in plasma indicated dysregulation of protein digestion and absorption, salivary secretion, and regulation of actin cytoskeleton, along with significant metabolic suppression. C4BPA, Caprolactam, Col15A1, and HBB were identified as predictors of stroke occurrence, and the Support Vector Machines (SVM) model was determined to be the optimal predictive model by integrating six machine-learning classification models. The SVM model showed strong performance in both the internal validation set (area under the curve [AUC]: 0.977, 95% confidence interval [CI]: 0.941-1.000) and the external independent validation set (AUC: 0.973, 95% CI: 0.921-0.999).

Construction and validation of a line chart for gestational diabetes mellitus based on clinical indicators

BackgroundGestational diabetes mellitus (GDM) is a common complication of mid–to-late pregnancy. Here, we constructed a predictive model for GDM based on a combination of clinical characteristics and relevant serum markers.MethodsData from full-term singleton vaginal deliveries from January 2022 to January 2023 were retrospectively collected from the obstetrics department. The data collected were segregated and assigned to training, validation, and external test sets. Maternal demographic characteristics, living and working habits, and haematological indicators, such as liver function and lipids were collected using a questionnaire designed for the study. The “rms” package in R was used to explore GDM-associated factors through stepwise regression at P < 0.05. A predictive model was developed based on the results of multifactorial logistic regression analysis. We then evaluated the differentiation of the column-line graphical model and performed internal and external validation. To assess the accuracy of the bar graphical model, we plotted calibration and decision curves.ResultsData from 265 pregnant women were included in the training and internal validation sets, and data from 113 pregnant women were included in the external validation set. The logistic regression algorithm screened 8 indicators as predictors. A prediction model was constructed with ALT, TBA, TC, and TG levels while considering whether GDM affects appetite, the husband– wife relationship, family history, and parental relationships as predictors. The Hosmer–Lemeshow goodness-of-fit test revealed that the chi-square values for the modelling, internal validation, and external validation groups (χ2 = 5.964, 3.249, and 12.182, respectively) were all P > 0.05. The ROC curve AUCs for the three groups were 0.93 (95% CI: 0.89–0.97), 0.72 (95% CI: 0.62–0.81), and 0.68 (95% CI: 0.53–0.83), respectively.ConclusionIn this study, a GDM prediction model was constructed to achieve high performance in GDM risk prediction based on routine obstetric tests and information.

Automated quantification of cerebral microbleeds in susceptibility-weighted MRI: association with vascular risk factors, white matter hyperintensity burden, and cognitive function.

To train and validate a deep learning (DL)-based segmentation model for cerebral microbleeds (CMB) on susceptibility-weighted MRI; and to find associations between CMB, cognitive impairment, and vascular risk factors. Participants in this single-institution retrospective study underwent brain MRI to evaluate cognitive impairment between January-September 2023. For training the DL model, the nnU-Net framework was used without modifications. The DL model's performance was evaluated on independent internal and external validation datasets. Linear regression analysis was used to find associations between log-transformed CMB numbers, cognitive function (mini-mental status examination [MMSE]), white matter hyperintensity (WMH) burden, and clinical vascular risk factors (age, sex, hypertension, diabetes, lipid profiles, and body mass index). Training of the DL model (n = 287) resulted in a robust segmentation performance with an average dice score of 0.73 (95% CI, 0.67-0.79) in an internal validation set, (n = 67) and modest performance in an external validation set (dice score = 0.46, 95% CI, 0.33-0.59, n = 68). In a temporally independent clinical dataset (n = 448), older age, hypertension, and WMH burden were significantly associated with CMB numbers in all distributions (total, lobar, deep, and cerebellar; all P <.01). MMSE was significantly associated with hyperlipidemia (β = 1.88, 95% CI, 0.96-2.81, P <.001), WMH burden (β = -0.17 per 1% WMH burden, 95% CI, -0.27-0.08, P <.001), and total CMB number (β = -0.01 per 1 CMB, 95% CI, -0.02-0.001, P = .04) after adjusting for age and sex. The DL model showed a robust segmentation performance for CMB. In all distributions, CMB had significant positive associations with WMH burden. Increased WMH burden and CMB numbers were associated with decreased cognitive function. CMB = cerebral microbleed; DL = deep learning, DSC = dice similarity coefficient; MMSE = mini-mental status examination; SVD = small vessel disease; SWI = susceptibility-weighted image; WMH = white matter hyperintensity.

Development and validation of a multivariable nomogram predictive of hepatitis B e antigen seroconversion after pregnancy in hepatitis B virus-infected mothers.

Current guidelines are controversial regarding the continuation of nucleos(t)ide analogues (NAs) therapy after delivery in Hepatitis B virus (HBV)-infected pregnant women. The postpartum period may be an opportune moment for achieving hepatitis B e antigen (HBeAg) seroconversion earlier with constant NAs therapy due to the restoration of immune function after delivery. We investigated prenatal and pregnant factors associated with HBeAg seroconversion after pregnancy and developed a nomogram to predict HBeAg seroconversion rates, aiding decision-making for optimal management in women. We retrospectively included 489 HBeAg-positive mothers as the training cohort from January 2014 to December 2018 and prospectively enrolled 94 patients as the external validation cohort from January 2019 to December 2021 at the First Affiliated Hospital of Xi'an Jiaotong University. In the training cohort, independent predictors were identified using the least absolute shrinkage and selection operator (LASSO) regression algorithm. Subsequently, multivariate logistic regression was employed to establish the nomogram. Model performance was assessed using the area under the receiver operating characteristic curve (AUC), calibration plots, and decision curve analysis (DCA). Both discrimination and calibration were evaluated through bootstrapping with 1,000 resamples. The external validation cohort was subsequently used to validate the nomogram. Factors such as pregnancy hepatitis flare (OR: 5.122, 95% CI: 2.725-9.928, p < 0.001), NAs therapy after delivery (OR: 15.051, 95% CI: 6.954-37.895, p: <0.001), hepatitis B surface antigen (HBsAg) (OR: 0.549, 95% CI: 0.366-0.812, p: 0.003) and HBV DNA level at delivery (OR: 0.785, 95% CI: 0.619-0.986, p: 0.041) were included in the final risk model. The AUC in the training set was 0.873 (95% CI: 0.839-0.904). The calibration curve of the nomogram closely resembled the ideal diagonal line. DCA showed a significantly better net benefit in the model. External validation also confirmed the reliability of the prediction nomogram. The AUC in the external validation set was 0.889 (95% CI: 0.801-0.953). The calibration curve for the external validation set was in close proximity to the ideal diagonal line. DCA also demonstrated a significant net benefit associated with the predictive model, consistent with the findings in the training set. Finally, the nomogram has been translated into an online risk calculator that is freely available to the public (https://wendyzhong.shinyapps.io/DynNomapp/). We developed a nomogram based on prenatal and pregnant factors to estimate HBeAg seroconversion after delivery in women. This tool provides clinicians with a precise and effective way to identify individuals likely to undergo HBeAg seroconversion postpartum, aiding in decision-making for optimal management.

Development and validation of a machine learning‐based model of ischemic stroke risk in the Chinese elderly hypertensive population

AbstractMachine learning (ML) has made some significant contributions to stroke prevention, but the stability and accuracy of existing models for clinical applications are uncertain. This study develops and validates an interpretable ML model using metabolic and coagulation biomarkers to predict ischemic stroke in elderly hypertensive patients in Northwest China. The prediction model used 453 electronic medical records for the model building (80% as a training set and 20% as a test set) and 132 for external validation. The final seven key features (D‐dimer, cystatin C, homocysteine, hemoglobin A1c, prothrombin time, low‐density lipoprotein C, and triglyceride glucose‐body mass index) were selected by the advanced approach, elastic net, and classical wrapping approaches. The final model, eXtreme gradient boosting, was identified as having superior performance than the other 9 classifers (random forest, Gaussian process, multilayer perceptron, logistic regression, support vector machine, K‐nearest neighbor, decision tree, Gaussian naive bayes, and ensemble model), with area under the receiver‐operating characteristic curves of 0.97 and 0.94 for the test and external validation sets, respectively. The final model demonstrates excellent stability, accuracy, and clinical usefulness through various metrics and decision curve analysis. Additionally, an online human–machine interface application has been developed for clinical practice to help early identification and intervention for ischemic stroke in elderly hypertensive patients.

Prediction of lymphovascular invasion in invasive breast cancer based on clinical-MRI radiomics features

ObjectiveWe aim to develop a predictive model for lymphovascular invasion (LVI) in patients with invasive breast cancer (IBC), using magnetic resonance imaging (MRI)-based radiomics features.MethodsA total of 204 patients with IBC admitted to our hospital were included in this retrospective study. The data was split into training and validation sets at a 7:3 ratio. Feature normalization was conducted, followed by feature selection using ANOVA, correlation analysis, and LASSO in the training set. The final step involved building a logistic regression model. The LVI prediction models were established by single sequence image and combined different sequence images as follows: A: prediction model based on the optimal sequence in the 7-phase enhanced MRI scans; B: prediction model based on the optimal sequences in the sequences T1WI, T2WI, and DWI; and C: the combined model based on the optimal sequences selected from A and B. Subjects' work characteristic curves (ROC) and decision curves (DCA) were plotted to determine the extent to which they predicted LVI performance in the training and validation sets. Simultaneously, nomogram models were constructed by integrating radiomics features and independent risk factors. In addition, an additional 16 patients from the center between January and August 2024 were collected as the Nomogram external validation set. The ROC and DCA were used to evaluate the performance of the model.ResultsIn the enhanced images, Model A built based on the enhanced 2-phase achieved the best average AUC, with a validation set of 0.764. Model B built based on the T2WI had better results, with a validation set of 0.693. Model C built by combining enhanced 2-phase and T2WI sequences had a mean AUC of 0.705 in the validation set. In addition, the tumor size, whether the tumor boundary was clear or not, and whether there was a coelom in the tumor tissue had a statistically significant effect on the LVI of IBC, and a clinical-radiomics nomogram was established. DCAs as well as Nomogram also indicate that Model A has good clinical utility. The AUC of the nomogram in the training set, internal validation set, and external validation set were 0.703, 0.615, and 0.609, respectively. The DCA also showed that the radiomics nomogram combined with clinical factors had good predictive ability for LVI.ConclusionIn IBC, MRI radiomics can serve as a noninvasive predictor of LVI. The clinical-MRI radiomics model, as an efficient visual prognostic tool, shows promise in forecasting LVI. This highlights the significant potential of pre-radiomics prediction in enhancing treatment strategies.

Identification of PANoptosis Subtypes to Assess the Prognosis and Immune Microenvironment of Lung Adenocarcinoma Patients: A Bioinformatics Combined Machine Learning Study.

PANoptosis, a novelty mechanism of cell death involving crosstalk between apoptosis, pyroptosis, and necroptosis, is strongly associated with tumor cell death and immunotherapy efficacy. However, its relevance in lung adenocarcinoma (LUAD) remains to be elucidated. In this study, we acquired 18 PANoptosis-related differentially expressed gene (PRDEG) of LUAD. Based on these genes, LUAD samples were identified with different sub-types by unsupervised clustering. Next, we compared the differences between the subtypes, including clinical features, immune microenvironment, and potentially sensitive drugs. Further-more, we used machine learning to identify hub prognostic PRDEGs, construct a risk score, and validate it on other external datasets. We incorporated the patient's clinical information and risk score into the proportional hazards model and lasso-cox models to find key prognostic features and constructed five prognostic models. The best model was identified via the area under the curve and validated on an external dataset. LUAD patients were divided into two clusters named C1 and C2, respectively. The C2 cluster exhibited shorter survival time, more advanced tumor stage, higher suppressive immune cell scores, such as dendritic cells, and higher expression of inhibitory immune checkpoints, such as LAG3 and CD86. TIMP1, CAV1, and CD69 were recognized as key prognostic factors, and risk scores predicted survival with significant differences in the external validation set. Risk score and N-stage were identified as critical prognostic features. The Coxph model outper-formed other machine learning clinical models. The 1-, 3-, and 5-year time-ROCs in the exter-nal validation set were 0.55, 0.59, and 0.60, respectively. We demonstrated the potential of PANoptosis-based molecular clustering and prognostic features in predicting the survival of patients with LUAD as well as the tumor mi-croenvironment.

Nomogram based on virtual hyperemic pullback pressure gradients for predicting the suboptimal post-PCI QFR outcome after stent implantation.

Growing evidence shows an association between higher post-PCI quantitative flow ratios (QFR) and improved clinical prognosis, however, no models are available to predict suboptimal QFRs (< 0.91) after angiographically successful PCI. This study aims to establish a prediction nomogram for this domain. This study included 450 vessels derived from 421 consecutive patients enrolled in the PIONEER IV trial, which were randomly assigned in a 1:1 ratio to a training (N = 225) and internal validation (N = 225) set, with external validation performed in 97 vessels from 95 consecutive patients enrolled in the ASET Japan trial. LASSO regression was used for optimal feature selection, and multivariate logistic regression was subsequently utilized to construct the nomogram. The performance of the nomograms was assessed and validated by area under the receiver operating characteristics curve (AUC), calibration curves, decision curve analysis, and clinical impact curves. The nomogram was constructed incorporating a novel metric, quantitative flow ratio derived pullback pressure gradient (QFR-PPG), alongside four conventional parameters: left anterior descending artery disease, pre-procedural QFR, reference vessel diameter, and percent diameter stenosis. AUCs of the nomogram were 0.866 (95%CI:0.818-0.914), 0.784 (95% CI:0.722-0.847), and 0.781 (95% CI:0.682-0.879) in the training, internal validation and external validation sets, respectively. Bias-corrected curves revealed a strong consistency between actual observations and prediction. The risk of a suboptimal post-PCI QFR in patients after angiographically successful PCI can be effectively predicted using a nomogram incorporating five variables available pre-PCI, with its performance and clinical predictive value confirming its utility in helping clinicians with decision-making and planning revascularization. Registered on clinicaltrial.gov (NCT04923191 and NCT05117866).

Development and external validation of a prediction model for interstitial lung disease in systemic lupus erythematosus patients: A cross-sectional study

ObjectiveThe aim of this study is to develop and validate a nomogram that can assist clinicians in identifying female systemic lupus erythematosus (SLE) patients of reproductive age complicated with interstitial lung disease (ILD). MethodsClinical, laboratory data of SLE patients were first collected. Meteorological data were then gathered according to the geographical locations of the SLE patients. Diagnostic results, univariate logistic regression, elastic net regression, and multivariate logistic regression were used to screen for risk factors for female SLE patients of reproductive age complicated with ILD. A nomogram was constructed using these risk factors and was internally and externally validated through methods such as calculating the concordance index, plotting calibration curves, drawing receiver operating characteristic curves, and clinical decision curves. ResultsA total of 4798 SLE patients were included in this study, with 2488 patients in the development set and 2310 patients in the external validation set. The patients in the development set were randomly divided into a training set (N = 1742) and an internal testing set (N = 746) at a ratio of 7:3. Eight independent risk factors for ILD were identified, including APOB, APOA1, ALP, PLT, HCT, EOS-R, LYM-R, and age. The nomogram model was developed, and the areas under the receiver operating characteristic curve was 0.811 (0.748, 0.875), 0.820 (0.727,0.913), and 0.889 (0.869, 0.909) for the three sets, respectively. ConclusionWe established a nomogram model using easily accessible clinical and laboratory data to predict the probability of female SLE patients of reproductive age developing ILD.

Application of artificial intelligence model in pathological staging and prognosis of clear cell renal cell carcinoma

This study aims to develop a deep learning (DL) model based on whole-slide images (WSIs) to predict the pathological stage of clear cell renal cell carcinoma (ccRCC). The histopathological images of 513 ccRCC patients were downloaded from The Cancer Genome Atlas (TCGA) database and randomly divided into training set and validation set according to the ratio of 8∶2. The CLAM algorithm was used to establish the DL model, and the stability of the model was evaluated in the external validation set. DL features were extracted from the model to construct a prognostic risk model, which was validated in an external dataset. The results showed that the DL model showed excellent prediction ability with an area under the curve (AUC) of 0.875 and an average accuracy score of 0.809, indicating that the model could reliably distinguish ccRCC patients at different stages from histopathological images. In addition, the prognostic risk model constructed by DL characteristics showed that the overall survival rate of patients in the high-risk group was significantly lower than that in the low-risk group (P = 0.003), and AUC values for predicting 1-, 3- and 5-year overall survival rates were 0.68, 0.69 and 0.69, respectively, indicating that the prediction model had high sensitivity and specificity. The results of the validation set are consistent with the above results. Therefore, DL model can accurately predict the pathological stage and prognosis of ccRCC patients, and provide certain reference value for clinical diagnosis.

Construction and validation of a diagnostic model for high-risk papillary thyroid microcarcinoma.

The purpose of this study was to construct a diagnostic model by exploring the potential predictors of high-risk Papillary Thyroid Microcarcinoma (PTMC) and verifying its reliability. A retrospective analysis of PTMC patients who underwent surgical treatment from 2004 to 2015 in the SEER database (training set) and the clinical pathological ultrasound information of PTMC patients at the Sichuan Provincial People's Hospital from 2020 to 2022 (external validation set) was conducted. In the training set, univariate and multivariate logistic regression analyses were used to screen independent predictive factors for high-risk PTMC patients in pathology. A nomogram diagnostic model was further constructed. Additionally, ROC curves and calibration curves were drawn to evaluate the efficiency of the model. In the external validation set, the diagnostic model was indirectly evaluated based on preoperative ultrasound imaging features to explore the feasibility and reliability of diagnosing high-risk PTMC through preoperative ultrasound imaging features. A total of 1628 patients were included in the training set, and 530 patients were included in the test set. The independent risk factors for pathological high-risk PTMC were sex, age, tumor maximum diameter, tumor invasive, and cervical lymph nodes (P<0.05). The C-index of the nomogram constructed based on these five factors was 0.947, with an optimal sensitivity of 96.7% and a specificity of 86.0%. The calibration curve showed that the model had high consistency. The area under the curve (AUC) value of the ROC curve for high-risk PTMC predicted by the risk score based on ultrasound features was 0.824 [95% CI (0.789, 0.860)], which was highly consistent with the risk score based on pathological features (κ= 0.758, P<0.05). Indirect evaluation of a high-risk PTMC diagnostic model based on preoperative ultrasound imaging features had high predictive efficiency and potential value for clinical application.

Automated Transformation of Unstructured Cardiovascular Diagnostic Reports into Structured Datasets Using Sequentially Deployed Large Language Models.

Rich data in cardiovascular diagnostic testing are often sequestered in unstructured reports, with the necessity of manual abstraction limiting their use in real-time applications in patient care and research. We developed a two-step process that sequentially deploys generative and interpretative large language models (LLMs; Llama2 70b and Llama2 13b). Using a Llama2 70b model, we generated varying formats of transthoracic echocardiogram (TTE) reports from 3,000 real-world echo reports with paired structured elements, leveraging temporal changes in reporting formats to define the variations. Subsequently, we fine-tuned Llama2 13b using sequentially larger batches of generated echo reports as inputs, to extract data from free-text narratives across 18 clinically relevant echocardiographic fields. This was set up as a prompt-based supervised training task. We evaluated the fine-tuned Llama2 13b model, HeartDx-LM, on several distinct echocardiographic datasets: (i) reports across the different time periods and formats at Yale New Haven Health System (YNHHS), (ii) the Medical Information Mart for Intensive Care (MIMIC) III dataset, and (iii) the MIMIC IV dataset. We used the accuracy of extracted fields and Cohen's Kappa as the metrics and have publicly released the HeartDX-LM model. The HeartDX-LM model was trained on randomly selected 2,000 synthetic echo reports with varying formats and paired structured labels, with a wide range of clinical findings. We identified a lower threshold of 500 annotated reports required for fine-tuning Llama2 13b to achieve stable and consistent performance. At YNHHS, the HeartDx-LM model accurately extracted 69,144 out of 70,032 values (98.7%) across 18 clinical fields from unstructured reports in the test set from contemporary records where paired structured data were also available. In older echo reports where only unstructured reports were available, the model achieved 87.1% accuracy against expert annotations for the same 18 fields for a random sample of 100 reports. Similarly, in expert-annotated external validation sets from MIMIC-IV and MIMIC-III, HeartDx-LM correctly extracted 201 out of 220 available values (91.3%) and 615 out of 707 available values (87.9%), respectively, from 100 randomly chosen and expert annotated echo reports from each set. We developed a novel method using paired large and moderate-sized LLMs to automate the extraction of unstructured echocardiographic reports into tabular datasets. Our approach represents a scalable strategy that transforms unstructured reports into computable elements that can be leveraged to improve cardiovascular care quality and enable research.

Using high-repeatable radiomic features improves the cross-institutional generalization of prognostic model in esophageal squamous cell cancer receiving definitive chemoradiotherapy

ObjectivesRepeatability is crucial for ensuring the generalizability and clinical utility of radiomics-based prognostic models. This study aims to investigate the repeatability of radiomic feature (RF) and its impact on the cross-institutional generalizability of the prognostic model for predicting local recurrence-free survival (LRFS) and overall survival (OS) in esophageal squamous cell cancer (ESCC) receiving definitive (chemo) radiotherapy (dCRT).MethodsNine hundred and twelve patients from two hospitals were included as training and external validation sets, respectively. Image perturbations were applied to contrast-enhanced computed tomography to generate perturbed images. Six thousand five hundred ten RFs from different feature types, bin widths, and filters were extracted from the original and perturbed images separately to evaluate RF repeatability by intraclass correlation coefficient (ICC). The high-repeatable and low-repeatable RF groups grouped by the median ICC were further analyzed separately by feature selection and multivariate Cox proportional hazards regression model for predicting LRFS and OS.ResultsFirst-order statistical features were more repeatable than texture features (median ICC: 0.70 vs 0.42–0.62). RFs from LoG had better repeatability than that of wavelet (median ICC: 0.70–0.84 vs 0.14–0.64). Features with smaller bin widths had higher repeatability (median ICC of 8–128: 0.65–0.47). For both LRFS and OS, the performance of the models based on high- and low-repeatable RFs remained stable in the training set with similar C-index (LRFS: 0.65 vs 0.67, p = 0.958; OS: 0.64 vs 0.65, p = 0.651), while the performance of the model based on the low-repeatable group was significantly lower than that based on the high-repeatable group in the external validation set (LRFS: 0.61 vs 0.67, p = 0.013; OS: 0.56 vs 0.63, p = 0.013).ConclusionsApplying high-repeatable RFs in modeling could safeguard the cross-institutional generalizability of the prognostic model in ESCC.Critical relevance statementThe exploration of repeatable RFs in different diseases and different types of imaging is conducive to promoting the proper use of radiomics in clinical research.Key PointsThe repeatability of RFs impacts the generalizability of the radiomic model.The high-repeatable RFs safeguard the cross-institutional generalizability of the model.Smaller bin width helps improve the repeatability of RFs.Graphical

Prediction and validation of pathologic complete response for locally advanced rectal cancer under neoadjuvant chemoradiotherapy based on a novel predictor using interpretable machine learning

BackgroundPrecise evaluation of pathological complete response (pCR) is essential for determining the prognosis of patients with locally advanced rectal cancer (LARC) undergoing neoadjuvant chemoradiotherapy (NCRT) and can offer clues for the selection of subsequent treatment strategies. Most current predictive models for pCR focus primarily on pre-treatment factors, neglecting the dynamic systemic changes that occur during neoadjuvant chemoradiotherapy, and are constrained by low accuracy and lack of integrity. PurposeThis study devised a novel predictor of pCR using dynamic alterations in systemic inflammation-nutritional marker indexes (SINI) during neoadjuvant therapy and developed a machine-learning model to predict pCR. MethodsTwo cohorts of patients with LARC from center one from 2012 to 2017 and from center two from 2020 to 2023 were integrated for analysis. This study compared dynamic changes in blood indexes before and after neoadjuvant therapy and surgical operation. A least absolute shrinkage and selection operator (LASSO) regression analysis was conducted to mitigate collinearity and identify key indexes, constructing the SINI. Univariate and multiple logistic regression analyses were used to identify the independent risk factors associated with pCR. Additionally, 10 machine learning algorithms were employed to develop predictive models to assess risk. The hyperparameters of the machine learning models were optimized using a random search and 10-fold cross-validation. The models were assessed by examining various metrics, including the area under the receiver operating characteristic curves (AUC), the area under the precision-recall curve (AUPRC), decision curve analysis, calibration curves, and the precision and accuracy of the internal and external validation cohorts. Additionally, Shapley's additive explanations (SHAP) were employed to interpret the machine learning models. ResultsThe study cohort comprised 677 patients from the center one and 224 patients from the center two. Six key indexes were identified, and a predictive index, SINI, was constructed. Univariate and multiple logistic regression analyses revealed that SINI, clinical T-stage, clinical N-stage, tumor size, and the distance from the anal verge were independent risk factors for pCR in patients with LARC following NCRT. The mean AUC value of the extreme gradient boosting (XGB) model in the 10-fold cross-validation of the training set was 0.877. The XGB model demonstrated superior performance in the internal and external validation sets. Specifically, in the internal test set, the XGB model achieved an AUC of 0.86, AUPRC of 0.707, accuracy of 0.82, and precision of 0.80. In the external validation set, the XGB model exhibited an AUC of 0.83, AUPRC of 0.702, accuracy of 0.81, and precision of 0.81. Additionally, the predictions generated by the XGB model were analyzed using SHAP. ConclusionThis study involved developing and validating an XGB model using SINI to predict pCR in patients with LARC. Besides, a SINI-based machine learning model shows promise in accurately predicting pCR following NCRT in patients with resectable LARC, offering valuable insights for personalized treatment approaches.

Intratumoral and peritumoral radiomics model for the preoperative prediction of cribriform component in invasive lung adenocarcinoma: a multicenter study.

This study aimed to investigate the predictive value of intratumoral and peritumoral radiomics model for the cribriform component (CC) of invasive lung adenocarcinoma (LUAD). The 144 patients with invasive LUAD from our center were randomly divided into training set (n = 100) and internal validation set (n = 44) in a ratio of 7:3, and 75 patients from center 2 were regarded as the external validation set. Clinical risk factors were examined using univariate and multivariate logistic regression to construct the clinical model. We extracted radiomics features from gross tumor volume (GTV), gross and peritumoral volume (GPTV), and peritumoral volume (PTV), respectively. Radiomics models were constructed with selected features. A combined model based on the optimal Radscore and clinically independent predictors was constructed, and its predictive performance was assessed by receiver operating characteristic curve (ROC), calibration curve, and decision curve analysis (DCA). The area under curves (AUCs) of the GTV model were 0.882 (95% CI 0.817-0.948), 0.794 (95% CI 0.656-0.932), and 0.766 (95% CI 0.657-0.875) in the training, internal validation, and external validation sets, and the PTV model had AUCs of 0.812 (95% CI 0.725-0.899), 0.749 (95% CI 0.597-0.902), and 0.670 (95% CI 0.543-0.798) in the training, internal validation, and external validation sets, respectively. However, the GPTV radiomics model showed better predictive performance compared with the GTV and PTV radiomics models, with the AUCs of 0.950 (95% CI 0.911-0.989), 0.844 (95% CI 0.728-0.959), and 0.815 (95% CI 0.713-0.917) in the training, internal validation and external validation sets, respectively. In the clinical model, tumor shape, lobulation sign and maximal diameter were the independent predictors of CC in invasive LUAD. The combined model including independent clinical predictors and GPTV-Radscore show the considerable instructive to clinical practice, with the AUCs of 0.954(95% CI 0.918-0.990), 0.861(95% CI 0.752-0.970), and 0.794(95% CI 0.690-0.898) in training, internal validation, and external validation sets, respectively. DCA showed that the combined model had good clinical value and correction effect. Radiomics model is a very powerful tool for predicting CC growth pattern in invasive LUAD and can help clinicians make the strategies of treatment and surveillance in patients with invasive LUAD.

B-122 Improving the Diagnostic Utility of Multiplexed Assays in Laboratory Medicine with Machine Learning

Abstract Background The application of machine learning (ML) and artificial intelligence (AI) in precision medicine is of great interest and continues to grow. ML approaches offer a flexible and data-centric methodology for modeling diagnostic and prognostic trends, capable of identifying complex non-linear patterns and relationships between predictor variables and outcomes within large data sets. Although ML holds great promise, there are several common pitfalls that must be avoided for ML models to provide clinical utility. In the current study, we provide a rigorous and systematic evaluation of the utility of ML algorithms to predict disease in several representative data sets generated via multiplexed biomarker assays. We compare five major classes of ML algorithms compared to more traditional combinatorial analysis (logistic regression) to offer perspective against the current gold standard. Methods We obtained five data sets from published literature that include measurements of multiplexed biomarkers in patients with obstructive sleep apnea, sepsis, and breast and colon cancers in comparison to healthy controls. These datasets were generated by bead-based and planar multiplexed immunoassays. We chose representative ML algorithms from the classes of support vector machines (SVM), random forests (RF), neural networks (NNet), and extreme gradient boosting (xgBoost) in comparison to logistic regression (LR). Datasets were randomly split into internal validation and external validation sets via 10 iterations of permutation analysis. We tuned and validated the models using 5-fold cross validation, then tested those on external subsets. We report the average of several model classification metrics and associated variances across all resampled data sets. Results For the classification of sepsis using inflammatory cytokines measured on the Luminex bead-based immunoassay, RF outperformed LR in external validation sets by a 9.8% increase in AUROC. For classification of breast cancer patients, xgBoost and RF performed best with 20.0% and 16.9% increases in AUROC, respectively, whereas for classification of colon cancer patients, NNet outperformed LR by 11.5%. Across all datasets, on average, all ML algorithms outperformed LR in the internal validation sets by a 7.9% (6.3% - 9.9%) increase in AUROC. However, in external validation sets, only xgBoost and SVM outperformed LR, where RF and NNet showed signs of overfitting. Conclusions Overall, certain ML algorithms show improvement over LR for diagnostic applications using multiplexed assays, by up to 20% increase in AUROC. Interpretation of ML results must be performed rigorously to prevent overly optimistic and unrealistic conclusions. In the context of laboratory medicine, our results showcase the utility of ML in clinical applications, while highlighting potential disadvantages of ML. This analysis underscores the need for establishing rigorous guidelines to support data analytics during the development of novel multiplexed assays, which is needed to advance precision medicine.

Performance analysis of a deep-learning algorithm to detect the presence of inflammation in MRI of sacroiliac joints in patients with axial spondyloarthritis

ObjectivesTo assess the ability of a previously trained deep-learning algorithm to identify the presence of inflammation on MRI of sacroiliac joints (SIJ) in a large external validation set of patients...

The clinical prognostic risk stratification system for HIV infected hepatocellular carcinoma

BackgroundPatients with human immunodeficiency virus (HIV) are more susceptible to liver cancer because of their compromised immune system. There is no specific prognostic model for HIV-infected hepatocellular carcinoma (HCC) patients. MethodsClinical data of 85 patients with HIV-infected HCC was divided into a 7:3 ratio for training and internal validation sets, while the data of 23 patients with HIV-infected HCC was served as the external validation set. Data of 275 HIV-negative HCC patients was considered as external HIV-negative validation set. Variables associated with overall survival (OS) in the training set were used to develop the HIV-infected HCC prognosis (HIHP) model. The model was tested in the internal and external validation sets. The predictive accuracy of the model was assessed with conventional HIV-negative HCC prognostic scoring systems. ResultsIn the training set, variables independently associated with OS in multivariable analysis were organ involvement and tumor number. The HIHP model demonstrated a significant association with OS in the training set, with a median OS of 13 months for low risk, 7 months for medium risk, and 3 months for high risk (p < 0.001). The HIHP model showed a significant association with OS, and exhibited greater discriminative abilities compared to conventional HIV-negative HCC prognostic models both in the internal and external validation sets. In the external HIV-negative validation set, the HIHP model did not show better discrimination than conventional HIV-negative HCC scores. ConclusionThe new model presented in the work provided a more accurate prognostic prediction of OS in HIV-infected HCC patients. However, the model is not applicable to patients with HIV-negative HCC.

Development and external validation of a machine-learning based model to predict pre-sarcopenia in MASLD population: Results from NHANES 2017–2018

Introduction and ObjectivesWith rising prevalence of pre-sarcopenia in metabolic dysfunction-associated steatotic liver disease (MASLD), this study aimed to develop and validate machine learning-based model to identify pre-sarcopenia in MASLD population. Materials and MethodsA total of 571 MASLD subjects were screened from the National Health and Nutrition Examination Survey 2017–2018. This cohort was randomly divided into training set and internal testing set with a ratio of 7:3. Sixty-six MASLD subjects were collected from our institution as external validation set. Four binary classifiers, including Random Forest (RF), support vector machine, and extreme gradient boosting and logistic regression, were fitted to identify pre-sarcopenia. The best-performing model was further validated in external validation set. Model performance was assessed in terms of discrimination and calibration. Shapley Additive explanations were used for model interpretability. ResultsThe pre-sarcopenia rate was 17.51 % and 15.16 % in NHANES cohort and external validation set, respectively. RF outperformed other models with area under receiver operating characteristic curve (AUROC) of 0.819(95 %CI: 0.749, 0.889). When six top-ranking features were retained as per variable importance, including weight-adjusted waist, sex, race, creatinine, education and alkaline phosphatase, a final RF model reached an AUROC being 0.824(0.737, 0.910) and 0.732(95 %CI: 0.529, 0.936) in internal and external validation sets, respectively. The model robustness was proved in sensitivity analysis. The calibration curve and decision curve analysis confirmed a good calibration capacity and good clinical usage. ConclusionsThis study proposed a user-friendly model using explainable machine learning algorithm to predict pre-sarcopenia in MASLD population. A web-based tool was provided to screening pre-sarcopenia in community and hospitalization settings.