Area Under The Curve Of Model Research Articles

Application of multimodal deep learning and multi-instance learning fusion techniques in predicting STN-DBS outcomes for Parkinson's disease patients

Parkinson's Disease (PD) is a progressive neurodegenerative disorder with substantial impact on patients' quality of life. Subthalamic nucleus deep brain stimulation (STN-DBS) is an effective treatment for advanced PD, but patient responses vary, necessitating predictive models for personalized care. Recent advancements in medical imaging and machine learning offer opportunities to enhance predictive accuracy, particularly through deep learning and multi-instance learning (MIL) techniques. This retrospective study included 127 PD patients undergoing STN-DBS. Medical records and imaging data were collected, and patients were categorized based on treatment outcomes. Advanced segmentation models were trained for automated region of interest (ROI) delineation. A novel 2.5D deep learning approach incorporating multi-slice representation was developed to extract detailed ROI features. Multi-instance learning fusion techniques integrated predictions across multiple slices, combining radiomics and deep learning features to enhance model performance. Various machine learning algorithms were evaluated, and model robustness was assessed using cross-validation and hyperparameter optimization. The MIL model achieved an area under the curve (AUC) of 0.846 for predicting STN-DBS outcomes, surpassing the radiomics model's AUC of 0.825. Integration of MIL and radiomics features in the DLRad model further improved discriminative ability to an AUC of 0.871. Calibration tests showed good model reliability, and decision curve analysis demonstrated clinical utility, affirming the model's predictive advantage. This study demonstrates the efficacy of integrating MIL, radiomics, and deep learning techniques to predict STN-DBS outcomes in PD patients. The multimodal fusion approach enhances predictive accuracy, supporting personalized treatment planning and advancing patient care.

Criticality of Nursing Care for Patients With Alzheimer's Disease in the ICU: Insights From MIMIC III Dataset.

Alzheimer's disease (AD) patients admitted to intensive care units (ICUs) exhibit varying survival outcomes due to the unique challenges in managing AD patients. Stratifying patient mortality risk and understanding the criticality of nursing care are important to improve the clinical outcomes of AD patients. This study aimed to leverage machine learning (ML) and electronic health records (EHRs) only consisting of demographics, disease history, and routine lab tests, with a focus on nursing care, to facilitate the optimization of nursing practices for AD patients. We utilized Medical Information Mart for Intensive Care III, an open-source EHR dataset, and AD patients were identified based on the International Classification of Diseases, Ninth Revision codes. From a cohort of 453 patients, a total of 60 features, encompassing demographics, laboratory tests, disease history, and number of nursing events, were extracted. ML models, including XGBoost, random forest, logistic regression, and multi-layer perceptron, were trained to predict the 30-day mortality risk. In addition, the influence of nursing care was analyzed in terms of feature importance using values calculated from both the inherent XGBoost module and the SHapley Additive exPlanations (SHAP) library. XGBoost emerged as the lead model with a high accuracy of 0.730, area under the curve (AUC) of 0.750, sensitivity of 0.688, and specificity of 0.740. Feature importance analyses using inherent XGBoost module or SHAP both indicated the number of nursing care within 14 days post-admission as an important denominator for 30-day mortality risk. When nursing care events were excluded as a feature, stratifying patient mortality risk was also possible but the model's AUC of receiver operating characteristic curve was reduced to 0.68. Nursing care plays a pivotal role in the survival outcomes of AD patients in ICUs. ML models can be effectively employed to predict mortality risks and underscore the importance of specific features, including nursing care, in patient outcomes. Early identification of high-risk AD patients can aid in prioritizing intensive nursing care, potentially improving survival rates.

Construction of a molecular diagnostic system for neurogenic rosacea by combining transcriptome sequencing and machine learning

Patients with neurogenic rosacea (NR) frequently demonstrate pronounced neurological manifestations, often unresponsive to conventional therapeutic approaches. A molecular-level understanding and diagnosis of this patient cohort could significantly guide clinical interventions. In this study, we amalgamated our sequencing data (n = 46) with a publicly accessible database (n = 38) to perform an unsupervised cluster analysis of the integrated dataset. The eighty-four rosacea patients were partitioned into two distinct clusters. Neurovascular biomarkers were found to be elevated in cluster 1 compared to cluster 2. Pathways in cluster 1 were predominantly involved in neurotransmitter synthesis, transmission, and functionality, whereas cluster 2 pathways were centered on inflammation-related processes. Differential gene expression analysis and WGCNA were employed to delineate the characteristic gene sets of the two clusters. Subsequently, a diagnostic model was constructed from the identified gene sets using linear regression methodologies. The model's C index, comprising genes PNPLA3, CUX2, PLIN2, and HMGCR, achieved a remarkable value of 0.9683, with an area under the curve (AUC) for the training cohort's nomogram of 0.9376. Clinical characteristics from our dataset (n = 46) were assessed by three seasoned dermatologists, forming the NR validation cohort (NR, n = 18; non-neurogenic rosacea, n = 28). Upon application of our model to NR diagnosis, the model's AUC value reached 0.9023. Finally, potential therapeutic candidates for both patient groups were predicted via the Connectivity Map. In summation, this study unveiled two clusters with unique molecular phenotypes within rosacea, leading to the development of a precise diagnostic model instrumental in NR diagnosis.

Development and Validation of a Clinical Prediction Model for Diagnosing Mycoplasma Infections in Gynecological Patients.

In adult females, mycoplasma infection is common and challenging to diagnose. This study aimed to use retrospective laboratory data to construct a nomogram for predicting the mycoplasma infection of individuals with probable urogenital tract mycoplasma infection. A total of 2,859 patients with suspected urogenital tract mycoplasma infection were retrospectively enrolled in this study. Demographics and routine examinations of leucorrhea were used to develop a nomogram for predicting mycoplasma infection. The least absolute shrinkage and selection operator (LASSO) method was applied to filter variables and select predictors, and multivariable logistic regression was used to construct a nomogram. The discriminatory ability of the model was determined by calculating the area under the curve (AUC). The performance and clinical utility of the nomogram were generated by using Harrell's concordance index, calibration curve, and decision curve analysis (DCA). By using the LASSO regression method, seven variables (age, white blood cell, epithetical cell, cleanliness, candidiasis vaginalis, sialidases, and leukocyte esterase) were chosen, and a nomogram was constructed using these variables. The prediction nomogram (0.676, 95% CI: 0.611 - 0.744) demonstrated a satisfactory performance. The prediction model's AUC was 0.679 (95% CI: 0.660 - 0.691). Furthermore, the DCA showed a good clinical net benefit based on the mycoplasma infection nomogram. A nomogram was created in this study, which included seven demographic and clinical characteristics of female patients. The nomogram could be of great value for the diagnosis of mycoplasma infection.

Development and validation of a prediction model for rebound hyperbilirubinemia: a Chinese neonatal cohort study.

Rebound hyperbilirubinemia (HBB) is still present in as high as 10% of newborn babies. However, the applicability of established prediction models for rebound HBB to Chinese newborns is unclear. This study aimed to establish a model to predict HBB rebound after phototherapy among Chinese neonates. A retrospective cohort study was conducted on 1,035 HBB infants receiving phototherapy. Rebound HBB was defined as total serum bilirubin (TSB) returning to or above the American Academy of Pediatrics (AAP) phototherapy threshold within 72 hours after the end of phototherapy. The predictive effects of previously published two- and three-variable scores were verified. Neonates were randomly assigned in a 6:4 ratio to the training (n=621) group and the testing (n=414) group. All variables in the training set were used to select predictors by least absolute shrinkage and selection operator (LASSO) regression analysis. The internal validation of the prediction model was performed using the testing set. The model's predictive performance was evaluated by area under the curve (AUC), accuracy, sensitivity, and specificity, each with 95% confidence intervals (CIs). Receiver operating characteristic (ROC) and calibration curves were constructed to evaluate the discrimination ability and fitting effect of the prediction model, respectively. Rebound HBB was observed in 210 patients (20.3%). The AUC for the two- and three-variable scores were 0.498 (95% CI: 0.455-0.540) and 0.498 (95% CI: 0.457-0.540), respectively. Predictive factors for the risk of rebound HBB included formula feeding (>3 times/day), standard phototherapy irradiation time, TSB levels and age at termination of phototherapy, neonatal weight, and differences between TSB levels at the phototherapy termination and phototherapy threshold. The prediction model's AUC was 0.935 (95% CI: 0.911-0.958), the sensitivity was 0.880 (95% CI: 0.809-0.950), the specificity was 0.831 (95% CI: 0.790-0.871), and the accuracy was 0.841 (95% CI: 0.805-0.876). The established model performed well in predicting rebound risk among Chinese infants with HBB, which may be beneficial in treating and managing HBB in infants.

Machine learning models for differential diagnosing HER2-low breast cancer: A radiomics approach.

To develop machine learning models based on preoperative dynamic enhanced magnetic resonance imaging (DCE-MRI) radiomics and to explore their potential prognostic value in the differential diagnosis of human epidermal growth factor receptor 2 (HER2)-low from HER2-positive breast cancer (BC). A total of 233 patients with pathologically confirmed invasive breast cancer admitted to our hospital between January 2018 and December 2022 were included in this retrospective analysis. Of these, 103 cases were diagnosed as HER2-positive and 130 cases were HER2 low-expression BC. The Synthetic Minority Oversampling Technique is employed to address the class imbalance problem. Patients were randomly split into a training set (163 cases) and a validation set (70 cases) in a 7:3 ratio. Radiomics features from DCE-MRI second-phase imaging were extracted. Z-score normalization was used to standardize the radiomics features, and Pearson's correlation coefficient and recursive feature elimination were used to explore the significant features. Prediction models were constructed using 6 machine learning algorithms: logistic regression, random forest, support vector machine, AdaBoost, decision tree, and auto-encoder. Receiver operating characteristic curves were constructed, and predictive models were evaluated according to the area under the curve (AUC), accuracy, sensitivity, and specificity. In the training set, the AUC, accuracy, sensitivity, and specificity of all models were 1.000. However, in the validation set, the auto-encoder model's AUC, accuracy, sensitivity, and specificity were 0.994, 0.976, 0.972, and 0.978, respectively. The remaining models' AUC, accuracy, sensitivity, and specificity were 1.000. The DeLong test showed no statistically significant differences between the machine learning models in the training and validation sets (Z = 0, P = 1). Our study investigated the feasibility of using DCE-MRI-based radiomics features to predict HER2-low BC. Certain radiomics features showed associations with HER2-low BC and may have predictive value. Machine learning prediction models developed using these radiomics features could be beneficial for distinguishing between HER2-low and HER2-positive BC. These noninvasive preoperative models have the potential to assist in clinical decision-making for HER2-low breast cancer, thereby advancing personalized clinical precision.

A Comprehensive Model Outperformed the Single Radiomics Model in Noninvasively Predicting the HER2 Status in Patients with Breast Cancer

This study aimed to develop predictive models based on conventional magnetic resonance imaging (cMRI) and radiomics features for predicting human epidermal growth factor receptor 2 (HER2) status of breast cancer (BC) and compare their performance. A total of 287 patients with invasive BC in our hospital were retrospectively analyzed. All patients underwent preoperative breast MRI consisting of fat-suppressed T2-weighted imaging, axial dynamic contrast-enhanced MRI, and diffusion-weighted imaging sequences. From these sequences, radiomics features were derived. Three distinct models were established utilizing cMRI features, radiomics features, and a comprehensive model that amalgamated both. The predictive capabilities of these models were assessed using the receiver operating characteristic curve analysis. The comparative performance was then determined through the DeLong test and net reclassification improvement (NRI). In a randomized split, the 287 patients with BC were allotted to either training (234; 46 HER2-zero, 107 HER2-low, 81 HER2-positive) or test (53; 8 HER2-zero, 27 HER2-low, 18 HER2-positive) at an 8:2 ratio. The mean area under the curve (AUCs) for cMRI, radiomics, and comprehensive models predicting HER2 status were 0.705, 0.819, and 0.859 in training set and 0.639, 0.797, and 0.842 in test set, respectively. DeLong's test indicated that the combined model's AUC surpassed the radiomics model significantly (p<0.05). NRI analysis verified superiority of the combined model over the radiomics for BC HER2 prediction (NRI 25.0) in the test set. The comprehensive model based on the combination of cMRI and radiomics features outperformed the single radiomics model in noninvasively predicting the three-tiered HER2 status in patients with BC.

The role of genetic variants in the prediction of hearing loss due to cisplatin chemoradiotherapy.

Concomitant high-dose cisplatin with radiotherapy is commonly used for treating head and neck squamous cell carcinoma (HNSCC). Cisplatin, often used with radiotherapy, is known for causing irreversible sensorineural hearing loss, with individual variability suggesting a genetic component. This study aims to enhance the predictive ability of the clinical prediction model for cisplatin-induced hearing loss (CIHL) in HNSCC patients, as outlined in Theunissen etal., by incorporating significant genetic variants. Conducted at the Netherlands Cancer Institute, this retrospective study included 74 patients treated between 1997 and 2011. Thirty-one SNPs that were previously associated with CIHL or other cisplatin-induced toxicities were identified and incorporated into the model. The primary outcome measured was the change in decibels at posttreatment 1-2-4 kHz hearing levels per additional minor allele of these SNPs, evaluated using linear mixed-effects regression models. The model's predictive accuracy was determined by the area under the curve (AUC) using 10-fold cross-validation. The rs2289669 SNP in the SLC47A1/MATE1 gene was linked to a significant 2.67 dB increase in hearing loss per allele (95% CI 0.49-4.86, p = 0.017). Incorporating rs2289669 improved the model's AUC from 0.78 to 0.83, a borderline significant improvement (p = 0.073). This study underscores the importance of the rs2289669 SNP in CIHL and demonstrates the potential of combining genetic and clinical data for enhanced predictive models in personalized treatment strategies.

Pediatric prognostic models predicting inhospital child mortality in resource-limited settings: An external validation study.

Prognostic models provide evidence-based predictions and estimates of future outcomes, facilitating decision-making, patient care, and research. A few of these models have been externally validated, leading to uncertain reliability and generalizability. This study aims to externally validate four models to assess their transferability and usefulness in clinical practice. The models include the respiratory index of severity in children(RISC)-Malawi model and three other models by Lowlavaar et al. The study used data from the Clinical Information Network (CIN) to validate the four models where the primary outcome was in-hospital mortality. 163,329 patients met eligibility criteria. Missing data were imputed, and the logistic function was used to compute predicted risk of in-hospital mortality. Models' discriminatory ability and calibration were determined using area under the curve(AUC), calibration slope, and intercept. The RISC-Malawi model had 50,669 pneumonia patients who met the eligibility criteria, of which the case-fatality ratio was 4406 (8.7%). Its AUCwas 0.77 (95% CI: 0.77-0.78), whereas the calibration slope was 1.04 (95% CI: 1.00 -1.06), and calibration intercept was 0.81 (95% CI: 0.77-0.84). Regarding the external validation of Lowlavaar et al. models, 10,782 eligible patients were included, with an in-hospital mortality rate of 5.3%. The primary model's AUC was 0.75 (95% CI: 0.72-0.77), the calibration slope was 0.78 (95% CI: 0.71-0.84), and the calibration intercept was 0.37 (95% CI: 0.28-0.46). All models markedly underestimated the risk of mortality. All externally validated models exhibited either underestimation or overestimation of the risk as judged from calibration statistics. Hence, applying these models with confidence in settings other than their original development context may not be advisable. Our findings strongly suggest the need for recalibrating these model to enhance their generalizability.

Machine Learning-Enabled 30-Day Readmission Model for Stroke Patients.

Background and Purpose: Hospital readmissions impose a substantial burden on the healthcare system. Reducing readmissions after stroke could lead to improved quality of care especially since stroke is associated with a high rate of readmission. The goal of this study is to enhance our understanding of the predictors of 30-day readmission after ischemic stroke and develop models to identify high-risk individuals for targeted interventions.Methods: We used patient-level data from electronic health records (EHR), five machine learning algorithms (random forest, gradient boosting machine, extreme gradient boosting–XGBoost, support vector machine, and logistic regression-LR), data-driven feature selection strategy, and adaptive sampling to develop 15 models of 30-day readmission after ischemic stroke. We further identified important clinical variables.Results: We included 3,184 patients with ischemic stroke (mean age: 71 ± 13.90 years, men: 51.06%). Among the 61 clinical variables included in the model, the National Institutes of Health Stroke Scale score above 24, insert indwelling urinary catheter, hypercoagulable state, and percutaneous gastrostomy had the highest importance score. The Model's AUC (area under the curve) for predicting 30-day readmission was 0.74 (95%CI: 0.64–0.78) with PPV of 0.43 when the XGBoost algorithm was used with ROSE-sampling. The balance between specificity and sensitivity improved through the sampling strategy. The best sensitivity was achieved with LR when optimized with feature selection and ROSE-sampling (AUC: 0.64, sensitivity: 0.53, specificity: 0.69).Conclusions: Machine learning-based models can be designed to predict 30-day readmission after stroke using structured data from EHR. Among the algorithms analyzed, XGBoost with ROSE-sampling had the best performance in terms of AUC while LR with ROSE-sampling and feature selection had the best sensitivity. Clinical variables highly associated with 30-day readmission could be targeted for personalized interventions. Depending on healthcare systems' resources and criteria, models with optimized performance metrics can be implemented to improve outcomes.

Predicting Response to Hypomethylating Agents in Patients with Myelodysplastic Syndromes (MDS) Using Artificial Intelligence (AI)

Introduction While the hypomethylating agents (HMAs) azacitidine (AZA) and decitabine (DAC) improve cytopenias and prolong survival in MDS patients (pts), response is not guaranteed. Timely identification of non-responders could prevent prolonged exposure to ineffective therapy, thereby reducing toxicities and costs. Currently no widely accepted clinical or genomic models exist to predict response or resistance to HMAs. We developed a clinical model to predict response or resistance to HMA after 90 days of initiating therapy based on changes in blood counts using time series analysis technology similar to the kind used in Apple's Siri or Google Assistant. In the setting of voice recognition, the sequence and context of words determines the meaning of a sentence; similarly, we hypothesized that the pattern of changes in MDS pts' blood counts would predict response or resistance early during treatment. Methods We screened a cohort of 107 pts with MDS (per 2016 WHO criteria) who received HMAs at our institution between February 2005 and July 2013 and had regular CBCs drawn during treatment. Mutations from a panel of 60 genes commonly mutated in myeloid malignancy were included. Responses were assessed after 6 months of therapy per International Working Group (IWG) 2006 criteria. Pts were divided randomly into training (80%) and validation (20%) cohorts. To address the potential for bias due to a small sample size, an oversampling algorithm was used to cluster similar pts based on their CBC data, Revised International Prognostic Scoring System (IPSS-R) score, and % bone marrow blasts at the time of diagnosis. CBC data from the first 90 days of treatment were fed into deep neural network (recurrent neural network) and decision tree algorithms, which were trained to predict whether pts would achieve a response (defined as complete remission (CR), partial remission (PR), or hematologic Improvement (HI)). Area under the curve (AUC) was used to assess model performance. Important features that impact the algorithm's predictions were extracted and plotted. Results 20747 unique data points were used, including CBC, clinical and genomic data. Among 107 pts, 61 (57.0%) received AZA only, 19 (17.8%) DAC only, 4 (3.7%) received both DAC and AZA, and 23 (21.5%) received HMA with an additional agent. Median age was 69 years (range: 37-100 years), and 27 (26.4%) were female. Forty pts (37.4%) were very low/low risk, 32 (29.9%) intermediate, 19 (17.8%) high, and 16 (14.9%) very high risk per IPSS-R. Responses included 23 (22.5%) CR, 2 (1.9%) marrow CR, 4 (3.9%) PR, and 20 (19.6%) HI. The most commonly mutated genes were ASXL1 (17.6%), TET2 (16.7%), SRSF2 (15.7%), SF3B1 (11.8%), RUNX1 (10.8%), STAG2(10.8%), and DNMT3A (10.8%). The median number of mutations per sample was 1 (range, 0-11), and 40 pts (39.2%) had &gt; 3 mutations per sample. When trained using absolute values and changes in CBC values, the model's AUC was 0.95 in the training cohort and 0.83 in the validation cohort. When the cohort was oversampled to 1000 pts, the validation cohort AUC increased to 0.89. Feature extraction algorithms identified increases in MCV and RDW during weeks 2-8 of treatment, increased proportion of lymphocytes, decreased proportion of monocytes, and increased platelet counts during weeks 6-8 as factors favoring response to HMA. The model provides personalized, patient-specific predictions that correlate with blood counts (Figure 1). Conclusions We describe a machine learning model that monitors changes in blood counts during therapy with HMA to predict response or resistance to HMA in MDS pts. Such a model can be used to develop novel trial designs wherein pts predicted to not respond after 90 days of HMA treatment could be assigned to an investigational agent. Conversely, it would help inform the decision to continue HMA therapy in pts predicted to respond. Increasing sample size with oversampling dramatically increased model accuracy; a larger cohort of pts treated at different institutions is currently under development. Disclosures Sekeres: Millenium: Membership on an entity's Board of Directors or advisory committees; Syros: Membership on an entity's Board of Directors or advisory committees; Celgene: Membership on an entity's Board of Directors or advisory committees. Mukherjee:Partnership for Health Analytic Research, LLC (PHAR, LLC): Consultancy; Takeda: Membership on an entity's Board of Directors or advisory committees; Celgene Corporation: Consultancy, Membership on an entity's Board of Directors or advisory committees, Research Funding; Projects in Knowledge: Honoraria; Novartis: Consultancy, Membership on an entity's Board of Directors or advisory committees, Research Funding; Pfizer: Honoraria; McGraw Hill Hematology Oncology Board Review: Other: Editor; Bristol-Myers Squibb: Speakers Bureau. Advani:Glycomimetics: Consultancy, Research Funding; Kite Pharmaceuticals: Consultancy; Amgen: Research Funding; Pfizer: Honoraria, Research Funding; Macrogenics: Research Funding; Abbvie: Research Funding. Maciejewski:Alexion: Consultancy; Novartis: Consultancy. Nazha:Novartis: Speakers Bureau; Tolero, Karyopharma: Honoraria; Abbvie: Consultancy; Jazz Pharmacutical: Research Funding; Incyte: Speakers Bureau; Daiichi Sankyo: Consultancy; MEI: Other: Data monitoring Committee.

Classical sheep scrapie in Great Britain: spatial analysis and identification of environmental and farm-related risk factors

BackgroundPrevious studies suggest that the spatial distribution of classical sheep scrapie in Great Britain is uneven and that certain flock characteristics may be associated with occurrence of the disease. However, the existence of areas of high and low disease-risk may also result from differences in the spatial distribution of environmental characteristics. In this study we explored the spatial pattern of classical scrapie in Great Britain between 2002 and 2005 and investigated the association between disease occurrence and various environmental and farm-related risk factors.ResultsExploratory spatial analysis: South Wales was found to have a higher density of scrapie-positive farms than the rest of Great Britain. In addition, a small cluster of high-risk farms was identified in the center of this region in which clustering of scrapie-positive farms occurred up to a distance of approximately 40 km.Spatial modelling: A mixed-effects regression model identified flock-size and soil drainage to be significantly associated with the occurrence of scrapie in England and Wales (area under the curve (AUC) 0.71 ± 0.01, 95% CI 0.68 - 0.74). The predictive risk map based on the estimated association between these factors and disease occurrence showed most of Wales to be at risk of being confirmed positive for scrapie with areas of highest risk in central and south Wales. In England, areas with the highest risk occurred mainly in the north and the midlands.ConclusionThe observed distribution of scrapie in Great Britain exhibited a definite spatial pattern with south Wales identified as an area of high occurrence. In addition both flock (flock size) and environmental variables (soil drainage) were found to be significantly associated with the occurrence of the disease. However, the model's AUC indicated unexplained variation remaining in the model and the source of this variation may lie in farm-level characteristics rather than spatially-varying ones such as environmental factors.