Area Under The ROC Curve Research Articles

Diagnostic Performance for Detection of Glaucomatous Structural Damage Using Pixelwise Analysis of Retinal Thickness Measurements.

To compare the diagnostic accuracy of thickness measurements of individual and combined macular retinal layers to discriminate 188 glaucomatous and 148 glaucoma suspect eyes from 362 healthy control (HC) eyes on a pixel-by-pixel basis. For this retrospective study, we manually corrected the segmentations of posterior pole optical coherence tomography (OCT) scans to determine the thickness of the nerve fiber layer (NFL), ganglion cell layer (GCL), inner plexiform layer (IPL), the ganglion cell complex (GCC), and the total neural retina (TR). For each eye, the total number of pixels with thickness values less than the fifth percentile of the HC distribution was used to create a receiver operating characteristic (ROC) curve for each layer and for layer combinations. Using total abnormal pixel count criteria to discriminate glaucoma from HC eyes, the individual layers with the highest area under the ROC curve (AUC) were the NFL and GCL; IPL performance was significantly lower (P < 0.05). GCC had a significant higher AUC (94.3%) than individual the AUC of the NFL (92.3%) (P = 0.0231) but not higher than AUC of the GCL (93.4%) (P = 0.3487). The highest AUC (95.4%) and sensitivity (85.1%) at 95% specificity was found for the Boolean combination of NFL or GCL. The highest AUC is not significantly higher (P = 0.0882) than the AUC of the GCC but the highest sensitivity is significantly higher than the sensitivity of the GCC. This pattern was similar for discriminating between suspect and HC eyes (P = 0.0356). Using pixel-based methods, the diagnostic accuracy of NFL and GCL exceeded that of IPL and TR. GCC had equivalent performance as NFL and GCL. The specific spatial locations within the posterior pole that exhibit best performance vary depending on which layer is being assessed. Recognizing this dependency highlights the importance of considering multiple layers independently, as they offer complementary information for effective and comprehensive diagnosis.

External validation and calibration of risk equations for prediction of diabetic kidney diseases among patients with type 2 diabetes in Taiwan

BackgroundMost existing risk equations for predicting/stratifying individual diabetic kidney disease (DKD) risks were developed using relatively dated data from selective and homogeneous trial populations comprising predominately Caucasian type 2 diabetes (T2D) patients. We seek to adapt risk equations for prediction of DKD progression (microalbuminuria, macroalbuminuria, and renal failure) using empiric data from a real-world population with T2D in Taiwan.MethodsRisk equations from three well-known simulation models: UKPDS-OM2, RECODe, and CHIME models, were adapted. Discrimination and calibration were determined using the area under the receiver operating characteristic curve (AUROC), a calibration plot (slope and intercept), and the Greenwood-Nam-D’Agostino (GND) test. Recalibration was performed for unsatisfactory calibration (p-value of GND test < 0.05) by adjusting the baseline hazards of risk equations to address risk variations among patients.ResultsThe RECODe equations for microalbuminuria and macroalbuminuria showed moderate discrimination (AUROC: 0.62 and 0.76) but underestimated the event risks (calibration slope > 1). The CHIME equation had the best discrimination for renal failure (AUROCs from CHIME, UKPDS-OM2 and RECODe: 0.77, 0.60 and 0.64, respectively). All three equations overestimated renal failure risk (calibration slope < 1). After rigorous updating, the calibration slope/intercept of the recalibrated RECODe for predicting microalbuminuria (0.87/0.0459) and macroalbuminuria (1.10/0.0004) risks as well as the recalibrated CHIME equation for predicting renal failure risk (0.95/-0.0014) were improved.ConclusionsRisk equations for prediction of DKD progression in real-world Taiwanese T2D patients were established, which can be incorporated into a multi-state simulation model to project and differentiate individual DKD risks for supporting timely interventions and health economic research.

Risk-Specific Training Cohorts to Address Class Imbalance in Surgical Risk Prediction.

Machine learning tools are increasingly deployed for risk prediction and clinical decision support in surgery. Class imbalance adversely impacts predictive performance, especially for low-incidence complications. To evaluate risk-prediction model performance when trained on risk-specific cohorts. This cross-sectional study performed from February 2024 to July 2024 deployed a deep learning model, which generated risk scores for common postoperative complications. A total of 109 445 inpatient operations performed at 2 University of Florida Health hospitals from June 1, 2014, to May 5, 2021 were examined. The model was trained de novo on separate cohorts for high-risk, medium-risk, and low-risk Common Procedure Terminology codes defined empirically by incidence of 5 postoperative complications: (1) in-hospital mortality; (2) prolonged intensive care unit (ICU) stay (≥48 hours); (3) prolonged mechanical ventilation (≥48 hours); (4) sepsis; and (5) acute kidney injury (AKI). Low-risk and high-risk cutoffs for complications were defined by the lower-third and upper-third prevalence in the dataset, except for mortality, cutoffs for which were set at 1% or less and greater than 3%, respectively. Model performance metrics were assessed for each risk-specific cohort alongside the baseline model. Metrics included area under the receiver operating characteristic curve (AUROC), area under the precision-recall curve (AUPRC), F1 scores, and accuracy for each model. A total of 109 445 inpatient operations were examined among patients treated at 2 University of Florida Health hospitals in Gainesville (77 921 procedures [71.2%]) and Jacksonville (31 524 procedures [28.8%]). Median (IQR) patient age was 58 (43-68) years, and median (IQR) Charlson Comorbidity Index score was 2 (0-4). Among 109 445 operations, 55 646 patients were male (50.8%), and 66 495 patients (60.8%) underwent a nonemergent, inpatient operation. Training on the high-risk cohort had variable impact on AUROC, but significantly improved AUPRC (as assessed by nonoverlapping 95% confidence intervals) for predicting mortality (0.53; 95% CI, 0.43-0.64), AKI (0.61; 95% CI, 0.58-0.65), and prolonged ICU stay (0.91; 95% CI, 0.89-0.92). It also significantly improved F1 score for mortality (0.42; 95% CI, 0.36-0.49), prolonged mechanical ventilation (0.55; 95% CI, 0.52-0.58), sepsis (0.46; 95% CI, 0.43-0.49), and AKI (0.57; 95% CI, 0.54-0.59). After controlling for baseline model performance on high-risk cohorts, AUPRC increased significantly for in-hospital mortality only (0.53; 95% CI, 0.42-0.65 vs 0.29; 95% CI, 0.21-0.40). In this cross-sectional study, by training separate models using a priori knowledge for procedure-specific risk classes, improved performance in standard evaluation metrics was observed, especially for low-prevalence complications like in-hospital mortality. Used cautiously, this approach may represent an optimal training strategy for surgical risk-prediction models.

Tree-based classification model for Long-COVID infection prediction with age stratification using data from the National COVID Cohort Collaborative.

We propose and validate a domain knowledge-driven classification model for diagnosing post-acute sequelae of SARS-CoV-2 infection (PASC), also known as Long COVID, using Electronic Health Records (EHRs) data. We developed a robust model that incorporates features strongly indicative of PASC or associated with the severity of COVID-19 symptoms as identified in our literature review. The XGBoost tree-based architecture was chosen for its ability to handle class-imbalanced data and its potential for high interpretability. Using the training data provided by the Long COVID Computation Challenge (L3C), which was a sample of the National COVID Cohort Collaborative (N3C), our models were fine-tuned and calibrated to optimize Area Under the Receiver Operating characteristic curve (AUROC) and the F1 score, following best practices for the class-imbalanced N3C data. Our age-stratified classification model demonstrated strong performance with an average 5-fold cross-validated AUROC of 0.844 and F1 score of 0.539 across the young adult, mid-aged, and older-aged populations in the training data. In an independent testing dataset, which was made available after the challenge was over, we achieved an overall AUROC score of 0.814 and F1 score of 0.545. The results demonstrated the utility of knowledge-driven feature engineering in a sparse EHR data and demographic stratification in model development to diagnose a complex and heterogeneously presenting condition like PASC. The model's architecture, mirroring natural clinician decision-making processes, contributed to its robustness and interpretability, which are crucial for clinical translatability. Further, the model's generalizability was evaluated over a new cross-sectional data as provided in the later stages of the L3C challenge. The study proposed and validated the effectiveness of age-stratified, tree-based classification models to diagnose PASC. Our approach highlights the potential of machine learning in addressing the diagnostic challenges posed by the heterogeneity of Long-COVID symptoms.

Predictive value of IGF-1/IGFBP-3 ratio for thyroid nodules in type 2 diabetic mellitus.

To explore the predictive value of the IGF-1/IGFBP-3 ratio for the presence of thyroid nodules in patients with type 2 diabetes mellitus (T2DM). This observational study prospectively enrolled patients with T2DM at the Second Hospital of Jilin University between May 2021 and January 2022. Thyroid nodule (TN) status was determined by ultrasonography. Receiver operating characteristic (ROC) curve analysis was performed to assess the predictive value of the serum IGF-1/IGFBP-3 molar ratio for TNs. Multivariable logistic regression analysis was conducted to identify risk factors for thyroid nodules in patients with T2DM. A total of 122 patients (mean age ± standard deviation: 52.57 ± 11.71 years; 74 males) were enrolled. 37.7% (n=46) of patients did not have TNs, while 62.3% (n=76) had TNs. The duration of diabetes, age, and HDL-C level were significantly higher in the T2DM group with TNs compared to the group without TNs (all P < 0.05). The area under the ROC curve (AUC) for the combination of IGF-1, IGFBP-3, and the serum IGF-1/IGFBP-3 molar ratio in predicting TNs in T2DM patients was 0.619 (P < 0.001). Additionally, multivariable logistic regression analysis revealed that the duration of diabetes, age, fasting plasma glucose (FPG), fasting insulin (FINS), thyroid-stimulating hormone (TSH), IGF-1, and IGFBP-3 levels were independent risk factors for thyroid nodules, while the serum IGF-1/IGFBP-3 molar ratio level was an independent protective factor for thyroid nodules in patients with T2DM (all P < 0.05). The combination of IGF-1, IGFBP-3, and the serum IGF-1/IGFBP-3 molar ratio may have a better predictive value for TNs in T2DM patients than using any single marker alone. The duration of diabetes, age, FPG, FINS, TSH, IGF-1, IGFBP-3, and the serum IGF-1/IGFBP-3 molar ratio levels were independently associated with thyroid nodules in patients with T2DM.

Metabolic profiling of blood plasma in depression in pregnant women during early pregnancy.

The study aimed to perform metabolic profiling of serum samples using liquid chromatography with mass spectroscopy (LC-MS) and to explore potential biomarkers of early trimester depression. Using the Edinburgh Postnatal Depression Scale (EPDS), participants were randomly divided into study and control groups. Serum metabolic profiles of the two groups were analysed by using LC-MS. Differential metabolite and pathway analysis were identified by using orthogonal projections to latent structure-discriminant analysis (OPLS-DA) and Kyoto Encyclopedia of Genes and Genomes (KEGG) enrichment analysis. Additionally, least absolute shrinkage and selection operator (LASSO) logistic and receiver operating characteristic (ROC) curve analyses were also conducted to explore potential biomarkers of antenatal depression (AD). The study included 41 participants, consisting of 16 subjects with AD and 25 controls. A total of 22 different metabolites were identified (p < .005), mainly affecting glycerophospholipid metabolism, linoleic acid metabolism, synthesis and degradation of ketone bodies, phenylalanine metabolism, and butanoate metabolism. The area under the ROC curve (AUC) for the LysoPC (24:0) was 0.858. This suggests that LysoPC (24:0) may be a potentially effective predictor of risk factors for AD. The study suggests that LysoPC (24:0) may be an effective and specific lipid biomarker for early trimester depression.

Estimating and Predicting the Rate of Kidney Function Decline over 10 years in the General Population.

We aimed to estimate the rate of kidney function decline over 10 years in the general population and develop a machine learning model to predict it. We used the JMDC database from 2012 to 2021, which includes company employees and their family members in Japan, where annual health checks are mandated for people aged 40-74 years. We estimated the slope (average change) of estimated glomerular filtration rate (eGFR) over a period of 10 years. Then, using the annual health-check results and prescription claims for the first five years from 2012 to 2016 as predictor variables, we developed an XGBoost model, evaluated its prediction performance with the root mean squared error (RMSE), R2, and area under the receiver operating characteristic curve (AUROC) for rapid decliners (defined as the slope <-3 ml/min/1.73 m2/year) using 5-fold cross validation, and compared these indicators with those of (i) the simple application of the eGFR slope from 2012 to 2016 and (ii) the adjusted linear regression model. We included 126,424 individuals (mean age, 45.2 years; male, 82.4%; mean eGFR, 79.0 ml/min/1.73 m2 in 2016). The mean slope was -0.89 (standard deviation, 0.96) ml/min/1.73 m2/year. The predictive performance of the XGBoost model (RMSE, 0.78; R2, 0.35; and AUROC, 0.89) was better than that of either the simple application of the eGFR slope from 2012 to 2016 (RMSE, 1.94; R2, -3.03; and AUROC, 0.79) or the adjusted linear regression model (RMSE, 0.81; R2, 0.30; and AUROC, 0.87). We estimated the rate of kidney function decline over 10 years in the general population as well as demonstrated that application of machine learning to annual health-check and claims data provides better predictive performance compared to traditional methods.

Risk for Suicide Attempts Assessed Using the Patient Health Questionnaire–9 Modified for Teens

Suicide is a leading cause of death in US youths. To assess whether screening with supplemental items 10 to 13 on the Patient Health Questionnaire-9 modified for teens (PHQ-9M) improves prediction of youth suicide attempts beyond the information provided by the first 9 items alone (the PHQ-9). This retrospective cohort study used a retrospective cohort of adolescents aged 12 to 17 years who were screened for depression in outpatient facilities within a pediatric health care system between January 1, 2016, and December 31, 2022, with up to 1 year of follow-up to assess the occurrence of suicidal behavior. Follow-up was completed on December 31, 2023. Screening with the PHQ-9M. This study developed and compared prediction using 3 Cox proportional hazards regression models (CR-9, CR-13, and CR-3) of subsequent suicide attempts, determined by the hospital's electronic health records up to 1 year following the last PHQ-9M screening. The CR-9 model used the PHQ-9 and the CR-13 model used all 13 items of PHQ-9M. The CR-3 model used the 3 most impactful variables selected from the 13 PHQ-9M items and PHQ-9 total score. All models were evaluated across 4 prediction horizons (30, 90, 180, and 365 days) following PHQ-9M screenings. Evaluation metrics were the area under the receiver operating characteristic curve (AUROC) and the area under the precision recall curve (AUPRC). Of 130 028 outpatients (65 520 [50.4%] male) with 272 402 PHQ-9M screenings, 549 (0.4%) had subsequent suicide attempts within 1 year following the PHQ-9M screening. The AUROC of the CR-9 model in the 365-day horizon was 0.77 (95% CI, 0.75-0.79); of the CR-13 model, 0.80 (95% CI, 0.78-0.82); and of the CR-3 model, 0.79 (95% CI, 0.76-0.81); the AUPRC of the CR-9 model was 0.02 (95% CI, 0.02-0.03); of the CR-13 model, 0.03 (95% CI, 0.02-0.03); and of the CR-3 model, 0.02 (95% CI, 0.02-0.03). The 3 most impactful items using adjusted hazard ratios were supplemental item 13 (lifetime suicide attempts; 3.06 [95% CI, 2.47-3.80]), supplemental item 10 (depressed mood severity in the past year; 2.99 [95% CI, 2.32-3.86]), and supplemental item 12 (serious suicidal ideation in the past month; 1.63 [95% CI, 1.25-2.12]). All of the models achieved higher AUROCs as prediction horizons shortened. In this cohort study of adolescent PHQ-9M screenings, the supplemental items on PHQ-9M screening improved prediction of youth suicide attempts compared with screening using the PHQ-9 across all prediction horizons, suggesting that PHQ-9M screening should be considered during outpatient visits to improve prediction of suicide attempts.

Revealing the impact of social circumstances on the selection of cancer therapy through natural language processing of social work notes.

We aimed to investigate the impact of social circumstances on cancer therapy selection using natural language processing to derive insights from social worker documentation. We developed and employed a Bidirectional Encoder Representations from Transformers (BERT) based approach, using a hierarchical multi-step BERT model (BERT-MS), to predict the prescription of targeted cancer therapy to patients based solely on documentation by clinical social workers. Our corpus included free-text clinical social work notes, combined with medication prescription information, for all patients treated for breast cancer at UCSF between 2012 and 2021. We conducted a feature importance analysis to identify the specific social circumstances that impact cancer therapy regimen. Using only social work notes, we consistently predicted the administration of targeted therapies, suggesting systematic differences in treatment selection exist due to non-clinical factors. The findings were confirmed by several language models, with GatorTron achieving the best performance with an area under the receiver operating characteristic curve (AUROC) of 0.721 and a Macro F1 score of 0.616. The UCSF BERT-MS model, capable of leveraging multiple pieces of notes, surpassed the UCSF-BERT model in both AUROC and Macro-F1. Our feature importance analysis identified several clinically intuitive social determinants of health that potentially contribute to disparities in treatment. Leveraging social work notes can be instrumental in identifying disparities in clinical decision-making. Hypotheses generated in an automated way could be used to guide patient-specific quality improvement interventions. Further validation with diverse clinical outcomes and prospective studies is essential. Our findings indicate that significant disparities exist among breast cancer patients receiving different types of therapies based on social determinants of health. Social work reports play a crucial role in understanding these disparities in clinical decision-making.

Early prediction of mortality and morbidities in VLBW preterm neonates using machine learning.

Predicting mortality and specific morbidities before they occur may allow for interventions that may improve health trajectories. Integrating key maternal and postnatal infant variables in the first 2 weeks of age into machine learning (ML) algorithms will reliably predict survival and specific morbidities in VLBW preterm infants. ML algorithms were developed to integrate 47 features for predicting mortality, bronchopulmonary dysplasia (BPD), neonatal sepsis, necrotizing enterocolitis (NEC), intraventricular hemorrhage (IVH), cystic periventricular leukomalacia (PVL), and retinopathy of prematurity (ROP). A retrospective cohort (n = 3341) was used to train and validate the models with a repeated 10-fold cross-validation strategy. These models were then tested on a separate cohort (n = 447) to evaluate the final model performance. Among the seven ML algorithms employed, tree-based ensemble models, specifically Random Forest (RF) and XGBoost, had the best performance metrics. The area under the receiver operating characteristic curve (AUROC) of sepsis with or without meningitis (0.73), NEC (0.73), BPD (0.71), and mortality (0.74) exceeded 0.7, while the area under Precision-Recall curve (AUPRC) for all outcomes was greater than the prevalence, demonstrating effective risk stratification in VLBW preterm infants. Our study demonstrates the potential of predictive analytics leveraging ML techniques in advancing precision medicine. Reliable prediction of adverse outcomes before they occur has the potential to institute interventions and possibly improve health trajectories in VLBW preterm infants. We used machine learning to develop and test predictive models for mortality and five major morbidities in VLBW preterm infants. Individualized prediction of outcomes and individualized interventions will advance Precision Medicine in Neonatology.

Machine learning-based diagnostic prediction of minimal change disease: model development study

Minimal change disease (MCD) is a common cause of nephrotic syndrome. Due to its rapid progression, early detection is essential; however, definitive diagnosis requires invasive kidney biopsy. This study aims to develop non-invasive predictive models for diagnosing MCD by machine learning. We retrospectively collected data on demographic characteristics, blood tests, and urine tests from patients with nephrotic syndrome who underwent kidney biopsy. We applied four machine learning algorithms—TabPFN, LightGBM, Random Forest, and Artificial Neural Network—and logistic regression. We compared their performance using stratified 5-repeated 5-fold cross-validation for the area under the receiver operating characteristic curve (AUROC) and the area under the precision-recall curve (AUPRC). Variable importance was evaluated using the SHapley Additive exPlanations (SHAP) method. A total of 248 patients were included, with 82 cases (33%) were diagnosed with MCD. TabPFN demonstrated the best performance with an AUROC of 0.915 (95% CI 0.896–0.932) and an AUPRC of 0.840 (95% CI 0.807–0.872). The SHAP methods identified C3, total cholesterol, and urine red blood cells as key predictors for TabPFN, consistent with previous reports. Machine learning models could be valuable non-invasive diagnostic tools for MCD.

Prediction of homologous recombination deficiency from routine histology with attention-based multiple instance learning in nine different tumor types.

Homologous recombination deficiency (HRD) is recognized as a pan-cancer predictive biomarker that potentially indicates who could benefit from treatment with PARP inhibitors (PARPi). Despite its clinical significance, HRD testing is highly complex. Here, we investigated in a proof-of-concept study whether Deep Learning (DL) can predict HRD status solely based on routine hematoxylin & eosin (H&E) histology images across nine different cancer types. We developed a deep learning pipeline with attention-weighted multiple instance learning (attMIL) to predict HRD status from histology images. As part of our approach, we calculated a genomic scar HRD score by combining loss of heterozygosity (LOH), telomeric allelic imbalance (TAI), and large-scale state transitions (LST) from whole genome sequencing (WGS) data of n = 5209 patients across two independent cohorts. The model's effectiveness was evaluated using the area under the receiver operating characteristic curve (AUROC), focusing on its accuracy in predicting genomic HRD against a clinically recognized cutoff value. Our study demonstrated the predictability of genomic HRD status in endometrial, pancreatic, and lung cancers reaching cross-validated AUROCs of 0.79, 0.58, and 0.66, respectively. These predictions generalized well to an external cohort, with AUROCs of 0.93, 0.81, and 0.73. Moreover, a breast cancer-trained image-based HRD classifier yielded an AUROC of 0.78 in the internal validation cohort and was able to predict HRD in endometrial, prostate, and pancreatic cancer with AUROCs of 0.87, 0.84, and 0.67, indicating that a shared HRD-like phenotype occurs across these tumor entities. This study establishes that HRD can be directly predicted from H&E slides using attMIL, demonstrating its applicability across nine different tumor types.

Conditional probabilistic diffusion model driven synthetic radiogenomic applications in breast cancer.

This study addresses the heterogeneity of Breast Cancer (BC) by employing a Conditional Probabilistic Diffusion Model (CPDM) to synthesize Magnetic Resonance Images (MRIs) based on multi-omic data, including gene expression, copy number variation, and DNA methylation. The lack of paired medical images and genomics data in previous studies presented a challenge, which the CPDM aims to overcome. The well-trained CPDM successfully generated synthetic MRIs for 726 TCGA-BRCA patients, who lacked actual MRIs, using their multi-omic profiles. Evaluation metrics such as Frechet's Inception Distance (FID), Mean Square Error (MSE), and Structural Similarity Index Measure (SSIM) demonstrated the CPDM's effectiveness, with an FID of 2.02, an MSE of 0.02, and an SSIM of 0.59 based on the 15-fold cross-validation. The synthetic MRIs were used to predict clinical attributes, achieving an Area Under the Receiver-Operating-Characteristic curve (AUROC) of 0.82 and an Area Under the Precision-Recall Curve (AUPRC) of 0.84 for predicting ER+/HER2+ subtypes. Additionally, the MRIs served to accurately predicted BC patient survival with a Concordance-index (C-index) score of 0.88, outperforming other baseline models. This research demonstrates the potential of CPDMs in generating MRIs based on BC patients' genomic profiles, offering valuable insights for radiogenomic research and advancements in precision medicine. The study provides a novel approach to understanding BC heterogeneity for early detection and personalized treatment.

Prediction of pathological complete response to chemotherapy for breast cancer using deep neural network with uncertainty quantification.

The I-SPY 2 trial is a national-wide, multi-institutional clinical trial designed to evaluate multiple new therapeutic drugs for high-risk breast cancer. Previous studies suggest that pathological complete response (pCR) is a viable indicator of long-term outcomes of neoadjuvant chemotherapy for high-risk breast cancer. While pCR can be assessed during surgery after the chemotherapy, early prediction of pCR before the completion of the chemotherapy may facilitate personalized treatment management to achieve an improved outcome. Notably, the acquisition of dynamic contrast-enhanced magnetic resonance (DCEMR) images at multiple time points during the I-SPY 2 trial opens up the possibility of achieving early pCR prediction. In this study, we investigated the feasibility of the early prediction of pCR to neoadjuvant chemotherapy using multi-time point DCEMR images and clinical data acquired in the I-SPY2 trial. The prediction uncertainty was also quantified to allow physicians to make patient-specific decisions on treatment plans based on the level of associated uncertainty. The dataset used in our study included 624 patients with DCEMR images acquired at 3 time points before the completion of the chemotherapy: pretreatment (T0), after 3 cycles of treatment (T1), and after 12 cycles of treatment (T2). A convolutional long short-term memory (LSTM) network-based deep learning model, which integrated multi-time point deep image representations with clinical data, including tumor subtypes, was developed to predict pCR. The performance of the model was evaluated via the method of nested 5-fold cross validation. Moreover, we also quantified prediction uncertainty for each patient through test-time augmentation. To investigate the relationship between predictive performance and uncertainty, the area under the receiver operating characteristic curve (AUROC) was assessed on subgroups of patients stratified by the uncertainty score. By integrating clinical data and DCEMR images obtained at three-time points before treatment completion, the AUROC reached 0.833 with a sensitivity of 0.723 and specificity of 0.800. This performance was significantly superior (p<0.01) to models using only images (AUROC=0.706) or only clinical data (AUROC=0.746). After stratifying the patients into eight subgroups based on the uncertainty score, we found that group #1, with the lowest uncertainty, had a superior AUROC of 0.873. The AUROC decreased to 0.637 for group #8, which had the highest uncertainty. The results indicate that our convolutional LSTM network-based deep learning model can be used to predict pCR earlier before the completion of chemotherapy. By combining clinical data and multi-time point deep image representations, our model outperforms models built solely on clinical or image data. Estimating prediction uncertainty may enable physicians to prioritize or disregard predictions based on their associated uncertainties. This approach could potentially enhance the personalization of breast cancer therapy.

NBCR-ac4C: A Deep Learning Framework Based on Multivariate BERT for Human mRNA N4-Acetylcytidine Sites Prediction.

N4-acetylcytidine (ac4C) plays a crucial role in regulating cellular biological processes, particularly in gene expression regulation and disease development. However, experiments to identify ac4C in a wet lab are time-consuming and costly, and the learning-based methods struggle to capture the underlying semantic knowledge and relations within sequences. To address this, we propose a deep learning approach called NBCR-ac4C based on pretrained models. Specifically, we employ Nucleotide Transformer and DNABERT2 to construct contextual embedding of nucleotide sequences, which effectively mine and express context relations between different features in the sequence. Convolutional neural network (CNN) and ResNet18 are then applied to further extract shallow and deep knowledge from context embedding. Depending on extensive experiments for the prediction of ac4C sites in nucleotide sequences, we observe that NBCR-ac4C outperforms general learning-based models. It achieves the highest accuracy (ACC) of 83.51% and an Area Under the Receiver Operating Characteristic Curve (AUROC) of 89.58% on an independent test set. Moreover, the proposed model, compared to the current state-of-the-art (SOTA) model LSA-ac4C, demonstrates higher ACC and AUROC by 0.81-3.7% and 0.05-1.58%, respectively. The data set and code are available on https://github.com/2103374200/NBCR to facilitate further discussion on NBCR-ac4C.

Diagnostic value of IFN-gamma in tuberculous pleural effusion

BackgroundSimple, rapid, and accurate diagnosis of tuberculous pleural effusion (TPE) remains challenging. This study aimed to determine the accuracy of IFN-γ in diagnosing TPE. MethodsWe quantified the expression of interferon-gamma (IFN-γ) in blood (B), adenosine deaminase (ADA), and IFN-γ in pleural effusions (PE) from 25 TPE patients and 31 non-TPE patients using a combination of immunological assays and flow cytometric analysis. The diagnostic performance of these three biomarkers was evaluated using receiver operating characteristic (ROC) curves. ResultsWe found that IFN-γ levels in blood and pleural fluid were higher in the TPE group than in the non-TPE group. The mean concentration of IFN-γ in pleural fluid of the TPE group was 3140.90 (1817.94, 6611.05) pg/mL, while that of the non-TPE group was 4.91 (0.69, 8.6) pg/mL), and the difference was statistically significant (z = 6.39, P < 0.001). The mean blood IFN-γ was 40.19 (16.45, 59.08) pg/mL in the TPE group and 2.76 (1.96, 6.02) pg/mL in the non-TPE group, which was statistically different (z = 5.12, P < 0.001). The area under the ROC curve (AUC) for pleural fluid IFN-γ, blood IFN-γ, and ADA were 0.999 (95 % CI: 0.994–1.00), 0.901 (95 % CI: 0.798–1.00) and 0.996 (95 % CI: 0.987–1.00), respectively. ConclusionThis study confirms that IFN-γ has high diagnostic validity in patients with TPE and can potentially be an excellent biomarker.

The diagnostic value and clinical relevance of high frequency ultrasound and shear wave elastography in systemic sclerosis: an observational monocentric study.

To unravel the features of skin involvement in patients with systemic sclerosis (SSc) by high frequency ultrasound (HFU) and shear wave elastography (SWE). To assess the ultrasound capabilities to distinguish SSc patients from healthy controls (HCs). We recruited a cohort of SSc patients in this cross-sectional study. HFU and SWE were used to quantify skin thickness and skin stiffness. Receiver operating characteristic (ROC) curve analysis was used to assess the diagnostic capabilities of ultrasound in SSc. The correlation analysis was used to evaluate the clinical relevance of ultrasound measurements in SSc. 20 consecutive SSc patients and 20 age-, gender- and body mass index-matched HCs were included. The skin thickness and stiffness were significantly greater in SSc patients compared with HCs. Patients with high disease activity had higher skin thickness and stiffness compared with patients with low disease activity. The area under the ROC curve (AUC) of the dorsum of middle fingers assessed by HFU was 0.847 (95% CI, 0.761-0.933). The AUC of the forearms and dorsum of hands assessed by SWE were 0.909 (95% CI, 0.829-0.989) and 0.879 (95% CI, 0.807-0.951). Further, the combined HFU and SWE tests displayed the best diagnostic performance with an AUC of 0.980 (95% CI, 0.939-1.000). A significant positive correlation between the ultrasound measurements and the modified Rodnan skin score (mRSS) was observed. The application of ultrasound can assist with disease diagnosis, it is necessary to develop a standard operating protocol to help with future implementation of ultrasound in SSc.

12524 Predictive Factors For Type 2 Diabetes Remission: Insights From A Digital Twin Intervention Program At 18 Months

Abstract Disclosure: P. Shamanna: None. M. Dharmalingam: None. A. Vadavi: None. A. Keshavamurthy: None. S. Bhonsley: None. M. Thajudeen: None. A. Balasubramanian: None. S.R. Joshi: None. Background and aims: Digital twin (DT) platform uses Artificial Intelligence (AI) and Internet of Things, to integrate multi-dimensional data for precision nutrition and health recommendations via the mobile app. Remission was defined as A1C levels less than 6.5% without medication for over 3 months. Evidence suggests that remission is more likely in individuals with early type 2 diabetes (T2D) who are not using insulin therapy and who can engage in significant weight loss. We evaluated the predictive value of baseline parameters on T2D remission, including duration of diabetes, HbA1c levels, diabetic medications as drug count, and CGM metrics, with the objective to improvise the patient selection strategies for remission Methods We analyzed T2D patients (n=209) who completed 18 months of digital twin intervention. Potential predictors were examined by logistic regression analyses, with model performance evaluated by accuracy, precision, recall, and Area under the Receiver Operating Characteristic Curve (AUC-ROC) metrics. Results The model achieved an accuracy of 78.57%, precision of 79.41%, recall of 93.10%, and AUC-ROC of 0.745. Significant inverse associations with remission were observed for the duration of diabetes (OR: 0.74), baseline HbA1c (OR: 0.76), and the number of anti-diabetic medications as oral drugs (OR: 0.77), indicating decreased odds of remission with increasing values. Conversely, positive associations were found for Time Below Range 2 at baseline (OR: 1.22) and HOMA2IR at baseline (OR: 1.24), suggesting increased odds of remission with higher values in these metrics. We describe the Receiver Operating Characteristic (ROC) curve for the diabetes remission prediction model, with the area under the curve (AUC) indicating the model's discriminatory ability. The duration of diabetes emerged as the most significant negative predictor, followed by baseline HbA1c and the number of diabetic drugs, while counterintuitive findings like the positive association of baseline HOMA2IR with remission suggest areas for further investigation. Conclusion We identified baseline parameters significantly influencing the likelihood of T2D remission in people undergoing digital twin intervention. The odds ratios indicate the effect sizes of these predictors, highlighting the importance of personalized, data-driven treatment approaches that potentially enhance remission outcomes. Further validation of these findings in larger external cohorts would be useful. Presentation: 6/1/2024

6818 Adherence Risk Management in Patients on Recombinant Human Growth Hormone Therapy, With Traffic Light Alerting Systems Tailored for Both Weekday and Weekend Adherence

Abstract Disclosure: S. Loche: Advisory Board Member; Self; SL has received advisory board fees from Merck KGaA, Darmstadt, Germany. Consulting Fee; Self; SL has received consultancy fees from Merck KGaA, Darmstadt, Germany. Speaker; Self; SL has received lecture fees from Merck KGaA, Darmstadt, Germany. P. van Dommelen: Consulting Fee; Self; PvD has a consultancy agreement with Merck KGaA, Darmstadt, Germany. V. Tornincasa: Employee; Self; VT is an employee of Ares Trading SA, an affiliate of Merck KGaA. L. Arnaud: Employee; Self; LA is an employee of Ares Trading SA, an affiliate of Merck KGaA. E. Koledova: Employee; Self; EK is an employee of Merck KGaA,Darmstadt, Germany. Stock Owner; Self; EK is an employee of Merck KGaA,Darmstadt, Germany. Background: The integration of data-driven strategies, such as clustering analysis with an alerting system, empowers healthcare professionals with actionable insights. In this study, we focused on clustering adherence patterns during the weekdays and weekends in the first 12 weeks of treatment. Our aim is to develop a data-driven clinical decision support system based on "traffic light" visualizations addressing adherence challenges during both weekdays and weekends in patients receiving Recombinant Human Growth Hormone Therapy. Data and Methods: Adherence data between March-November 2023 were extracted from the newly launched third generation of easypod™ device (EP3) and GrowZen Connect Next ecosystem. EP3 is the only connected device that delivers r-hGH and monitors real-time adherence to therapy. EP3 was perceived by health care professionals as more intuitive, comfortable, user-friendly, simpler, and easier to use than previous EP2. Patients with age 2-18 years at treatment start, a 7-day regimen and complete adherence data available between 1-12 weeks of treatment were selected. Patients’ adherence to therapy was represented using mean and standard deviation (SD) of adherence (%) and the interquartile range (IQR) of the timing of injections throughout the day in minutes aggregated by weekday (Monday-Thursday) and weekend day (Friday-Sunday). Cluster analysis was used to categorize adherence patterns using a Gaussian mixture model. Following a traffic lights-inspired visualization approach, the algorithm was set to create three clusters: high (green), medium (yellow), and low adherence (red). The area under the receiver operating characteristic curve (AUC-ROC) was used to find optimum thresholds for independent traffic lights. Results: Data for 232 patients (124 boys and 108 girls, median (Q25-Q75) age at treatment start was 10.5 (7.6-12.3)) were available. The most appropriate traffic light used the SD of adherence during the weekend, with an AUC-ROC value of 0.87. For the weekday, adherence-based traffic lights using optimum thresholds were &amp;gt;98% (SD&amp;lt;10.2), 88-98% (SD:10.2-22.8) and &amp;lt;88% (SD&amp;gt;22.8) for high, medium, and low adherence, respectively. For IQR of the timing of injections throughout the weekday, optimum thresholds were &amp;lt;38, 38-70 and &amp;gt;70 (min). For the weekend, optimum thresholds were &amp;gt;96% (SD&amp;lt;7.7), 86-96% (SD:7.7-21.5) and &amp;lt;86% (SD&amp;gt;21.5). For IQR of the timing of injections throughout the weekend, optimum thresholds were &amp;lt;56, 56-84 and &amp;gt;84 (min). Conclusions: Our research indicates that implementing a practical data-driven alert system, utilizing our proposed traffic-light coding, would empower healthcare practitioners to monitor patients' adherence to growth hormone therapy comprehensively. By understanding patient behavior during the weekdays and weekend, healthcare practitioners can tailor interventions for improved treatment outcomes. Presentation: 6/3/2024

Multicenter Development and Prospective Validation of eCARTv5: A Gradient Boosted Machine Learning Early Warning Score.

Early detection of clinical deterioration using machine learning early warning scores may improve outcomes. However, most implemented scores were developed using logistic regression, only underwent retrospective validation, and were not tested in important subgroups. Our objective was to develop and prospectively validate a gradient boosted machine model (eCARTv5) for identifying clinical deterioration on the wards. Multicenter retrospective and prospective observational study. Inpatient admissions to the medical-surgical wards at seven hospitals in three health systems for model development (2006-2022) and at 21 hospitals from three health systems for retrospective (2009-2023) and prospective (2023-2024) external validation. All adult patients hospitalized at each participating health system during the study years. None MEASUREMENTS AND MAIN RESULTS: Predictor variables (demographics, vital signs, documentation, and laboratory values) were used in a gradient boosted trees algorithm to predict intensive care unit transfer or death in the next 24 hours. The developed model (eCART) was compared to the Modified Early Warning Score (MEWS) and the National Early Warning Score (NEWS) using the area under the receiver operating characteristic curve (AUROC). The development cohort included 901,491 admissions, the retrospective validation cohort included 1,769,461 admissions, and the prospective validation cohort included 205,946 admissions. In retrospective validation, eCART had the highest AUROC (0.835; 95%CI 0.834, 0.835), followed by NEWS (0.766 (95%CI 0.766, 0.767)), and MEWS (0.704 (95%CI 0.703, 0.704)). eCART's performance remained high (AUROC ≥0.80) across a range of patient demographics, clinical conditions, and during prospective validation. We developed eCART, which performed better than the NEWS and MEWS retrospectively, prospectively, and across a range of subgroups. These results served as the foundation for Food and Drug Administration clearance for its use in identifying deterioration in hospitalized ward patients.