Area Under Precision-recall Curve Research Articles

Artificial intelligence based prediction model of in-hospital mortality among females with acute coronary syndrome: for the Jerusalem Platelets Thrombosis and Intervention in Cardiology (JUPITER-12) Study Group.

Despite ongoing efforts to minimize sex bias in diagnosis and treatment of acute coronary syndrome (ACS), data still shows outcomes differences between sexes including higher risk of all-cause mortality rate among females. Hence, the aim of the current study was to examine sex differences in ACS in-hospital mortality, and to implement artificial intelligence (AI) models for prediction of in-hospital mortality among females with ACS. All ACS patients admitted to a tertiary care center intensive cardiac care unit (ICCU) between July 2019 and July 2023 were prospectively enrolled. The primary outcome was in-hospital mortality. Three prediction algorithms, including gradient boosting classifier (GBC) random forest classifier (RFC), and logistic regression (LR) were used to develop and validate prediction models for in-hospital mortality among females with ACS, using only available features at presentation. A total of 2,346 ACS patients with a median age of 64 (IQR: 56-74) were included. Of them, 453 (19.3%) were female. Female patients had higher prevalence of NSTEMI (49.2% vs. 39.8%, p < 0.001), less urgent PCI (<2 h) rates (40.2% vs. 50.6%, p < 0.001), and more complications during admission (17.7% vs. 12.3%, p = 0.01). In-hospital mortality occurred in 58 (2.5%) patients [21/453 (5%) females vs. 37/1,893 (2%) males, HR = 2.28, 95% CI: 1.33-3.91, p = 0.003]. GBC algorithm outscored the RFC and LR models, with area under receiver operating characteristic curve (AUROC) of 0.91 with proposed working point of 83.3% sensitivity and 82.4% specificity, and area under precision recall curve (AUPRC) of 0.92. Analysis of feature importance indicated that older age, STEMI, and inflammatory markers were the most important contributing variables. Mortality and complications rates among females with ACS are significantly higher than in males. Machine learning algorithms for prediction of ACS outcomes among females can be used to help mitigate sex bias.

Self-Supervised Learning for Improved Optical Coherence Tomography Detection of Macular Telangiectasia Type 2

Deep learning image analysis often depends on large, labeled datasets, which are difficult to obtain for rare diseases. To develop a self-supervised approach for automated classification of macular telangiectasia type 2 (MacTel) on optical coherence tomography (OCT) with limited labeled data. This was a retrospective comparative study. OCT images from May 2014 to May 2019 were collected by the Lowy Medical Research Institute, La Jolla, California, and the University of Washington, Seattle, from January 2016 to October 2022. Clinical diagnoses of patients with and without MacTel were confirmed by retina specialists. Data were analyzed from January to September 2023. Two convolutional neural networks were pretrained using the Bootstrap Your Own Latent algorithm on unlabeled training data and fine-tuned with labeled training data to predict MacTel (self-supervised method). ResNet18 and ResNet50 models were also trained using all labeled data (supervised method). The ground truth yes vs no MacTel diagnosis is determined by retinal specialists based on spectral-domain OCT. The models' predictions were compared against human graders using accuracy, sensitivity, specificity, positive predictive value (PPV), negative predictive value (NPV), area under precision recall curve (AUPRC), and area under the receiver operating characteristic curve (AUROC). Uniform manifold approximation and projection was performed for dimension reduction and GradCAM visualizations for supervised and self-supervised methods. A total of 2636 OCT scans from 780 patients with MacTel and 131 patients without MacTel were included from the MacTel Project (mean [SD] age, 60.8 [11.7] years; 63.8% female), and another 2564 from 1769 patients without MacTel from the University of Washington (mean [SD] age, 61.2 [18.1] years; 53.4% female). The self-supervised approach fine-tuned on 100% of the labeled training data with ResNet50 as the feature extractor performed the best, achieving an AUPRC of 0.971 (95% CI, 0.969-0.972), an AUROC of 0.970 (95% CI, 0.970-0.973), accuracy of 0.898%, sensitivity of 0.898, specificity of 0.949, PPV of 0.935, and NPV of 0.919. With only 419 OCT volumes (185 MacTel patients in 10% of labeled training dataset), the ResNet18 self-supervised model achieved comparable performance, with an AUPRC of 0.958 (95% CI, 0.957-0.960), an AUROC of 0.966 (95% CI, 0.964-0.967), and accuracy, sensitivity, specificity, PPV, and NPV of 90.2%, 0.884, 0.916, 0.896, and 0.906, respectively. The self-supervised models showed better agreement with the more experienced human expert graders. The findings suggest that self-supervised learning may improve the accuracy of automated MacTel vs non-MacTel binary classification on OCT with limited labeled training data, and these approaches may be applicable to other rare diseases, although further research is warranted.

Imbalanced customer churn classification using a new multi-strategy collaborative processing method

The rapid advancement of big data and artificial intelligence heralds a dual-edged era of opportunities and challenges for the banking sector. Indeed, enhancing a model's capability to accurately classify imbalanced datasets represents a critical challenge within the field of customer churn prediction (CCP). In this paper, to address the challenges presented by the problem of imbalanced customer classification, a new multi-strategy collaborative processing method named IADASYN-FLCatBoost is proposed from dual perspectives: data and algorithm. At the data level, the traditional Adaptive Synthetic (ADASYN) sampling is improved, that is, the LOF (Local Outlier Factor) algorithm is introduced to eliminate outliers, and the classification features are specially processed to synthesize new minority class samples, thus an improved ADASYN (IADASYN) algorithm is obtained. At the algorithm level, the Focal Loss is embedded into the CatBoost ensemble learning framework to form a new Focal Loss-CatBoost (FLCatBoost) to make a focal-aware, cost-sensitive version of imbalanced customer churn prediction. Moreover, the empirical analysis is conducted in conjunction with the credit card customer dataset obtained from the Kaggle platform. The results of the staged comparison experiments show that the proposed method IADASYN-FLCatBoost in this paper shows the best prediction performance. Comparing the proposed method with 5 other imbalanced classification algorithms and 20 classifiers composed of classical sampling methods and ensemble learning algorithms, it is verified that the classification effect of the proposed method performs best, and the values of Recall, F1 score, G-mean and Area under Precision-Recall curve (AUPRC) have been significantly improved. In addition, further verification of the model also proves that the proposed method has certain generalizability and is still valid for other banks and customer churn datasets of other industries.

ProtT5 and random forests-based viscosity prediction method for therapeutic mAbs

Viscosity is a key characteristic of therapeutic antibodies for subcutaneous administration which requires low volume and high concentration formulations. It would be highly beneficial to accurately predict the viscosity of newly developed therapeutic antibodies in the early stages of development. In this work, a ProtT5-XL-UniRef50 (ProtT5) and Random Forests (RF)-based prediction method was proposed for accurately predicting the viscosity of monoclonal antibodies, with only corresponding sequences needed. Starting from the given heavy and light chain V-region sequences, corresponding features were first extracted from the ProtT5 pretrained model. Kernel principal analysis (Kernel-PCA) was then used for reducing the extracted 2048-D (1024-D for each sequence) feature vector to a reasonable level for efficient training of the RF-regressor. Then, the RF model was constructed on 40 commercially available therapeutic antibodies and tested with 3-folds cross-validation. Test results show that the model could reproduce the viscosity value at a high level (Pearson correlation coefficient (PCC) = 0.928). Performance on classifying high (>30 cP) and low (<30 cP) viscosity is much more satisfactory, the Accuracy (ACC) and the area under precision-recall curve (AUC) of the classification model from validation tests are 0.975 and 1.000, respectively. Compared to 5 existing state-of-the-art viscosity prediction methods, the proposed method performs best which would facilitate high concentration antibody viscosity high-throughput screening.

Use of machine learning techniques for identifying ischemic stroke instead of the rule-based methods: a nationwide population-based study

BackgroundMany studies have evaluated stroke using claims data; most of these studies have defined ischemic stroke using an operational definition following the rule-based method. Rule-based methods tend to overestimate the number of patients with ischemic stroke.ObjectivesWe aimed to identify an appropriate algorithm for identifying stroke by applying machine learning (ML) techniques to analyze the claims data.MethodsWe obtained the data from the Korean National Health Insurance Service database, which is linked to the Ilsan Hospital database (n = 30,897). The performance of prediction models (extreme gradient boosting [XGBoost] or gated recurrent unit [GRU]) was evaluated using the area under the receiver operating characteristic curve (AUROC), the area under precision–recall curve (AUPRC), and calibration curve.ResultsIn total, 30,897 patients were enrolled in this study, 3145 of whom (10.18%) had ischemic stroke. XGBoost, a tree-based ML technique, had the AUROC was 94.46% and AUPRC was 92.80%. GRU showed the highest accuracy (99.81%), precision (99.92%) and recall (99.69%).ConclusionsWe proposed recurrent neural network-based deep learning techniques to improve stroke phenotyping. This can be expected to produce rapid and more accurate results than the rule-based methods.

Improving dementia phenotype prediction with functional mapping of genetic data in non‐European ancestries

AbstractBackgroundMany common genetic variants have been associated with dementia, but their ability to predict dementia prospectively is unknown, especially in non‐European ancestral populations. We sought to optimize dementia risk prediction models with machine learning in individuals of diverse genetic ancestries.MethodSamples were selected from the University of California Los Angeles Health System with their electronic health records (EHRs) and linked genetic data. Dementia phenotype was defined using diagnosis codes and mapped phecodes. Polygenic risk scores (PRSs) were built using summary statistics of different population‐based genome‐wide association studies (GWASs). Independent significant and functionally mapped single nucleotide polymorphisms (SNPs) were identified using the FUMA tool. Logistic regression models were built with 1) APOE status, 2) a single PRS, or 3) multiple PRSs. The model with age and sex only was built as the baseline model. We trained LASSO models to determine dementia risk SNPs from weighted SNP‐level genetic data. We compared the predictive performance of these models after five‐fold cross‐validation with area under precision‐recall curve (AUPRC) and area under receiver operating characteristic (AUC). Finally, identified risk SNPs were mapped to genes for biological interpretation.ResultThere were 975 individuals included in the final Hispanic Latinx American ancestry sample, of which 9.44% (N = 92) were dementia cases. The model with LASSO‐selected functional mapped SNPs from multiple neurodegenerative diseases GWASs outperformed all other genetic risk score models (AUPRC: 0.39, AUROC: 0.87), resulting in a 19.6% increase in AUPRC from the best PRS model, and a 26.9% increase from the baseline model. The best‐performing LASSO model included nine SNPs, mostly located in the 19q13, 4p15, and 17q21 regions. Results of mapped genes not only revealed well‐known risk genes of dementia but identified risk genes from other neurodegenerative diseases.ConclusionWe demonstrated that building genetic risk scores with trans‐ancestry GWAS and selecting potential causal SNPs can be beneficial for dementia prediction in non‐European ancestry populations. Further training using machine learning methods with functionally mapped SNP‐level genetic data showed significant improvement in predictive power compared to PRS models. Genetic risks of multiple neurodegenerative diseases may contribute to the predisposition to dementia.

Weakly supervised segmentation models as explainable radiological classifiers for lung tumour detection on CT images

PurposeInterpretability is essential for reliable convolutional neural network (CNN) image classifiers in radiological applications. We describe a weakly supervised segmentation model that learns to delineate the target object, trained with only image-level labels (“image contains object” or “image does not contain object”), presenting a different approach towards explainable object detectors for radiological imaging tasks.MethodsA weakly supervised Unet architecture (WSUnet) was trained to learn lung tumour segmentation from image-level labelled data. WSUnet generates voxel probability maps with a Unet and then constructs an image-level prediction by global max-pooling, thereby facilitating image-level training. WSUnet’s voxel-level predictions were compared to traditional model interpretation techniques (class activation mapping, integrated gradients and occlusion sensitivity) in CT data from three institutions (training/validation: n = 412; testing: n = 142). Methods were compared using voxel-level discrimination metrics and clinical value was assessed with a clinician preference survey on data from external institutions.ResultsDespite the absence of voxel-level labels in training, WSUnet’s voxel-level predictions localised tumours precisely in both validation (precision: 0.77, 95% CI: [0.76–0.80]; dice: 0.43, 95% CI: [0.39–0.46]), and external testing (precision: 0.78, 95% CI: [0.76–0.81]; dice: 0.33, 95% CI: [0.32–0.35]). WSUnet’s voxel-level discrimination outperformed the best comparator in validation (area under precision recall curve (AUPR): 0.55, 95% CI: [0.49–0.56] vs. 0.23, 95% CI: [0.21–0.25]) and testing (AUPR: 0.40, 95% CI: [0.38–0.41] vs. 0.36, 95% CI: [0.34–0.37]). Clinicians preferred WSUnet predictions in most instances (clinician preference rate: 0.72 95% CI: [0.68–0.77]).ConclusionWeakly supervised segmentation is a viable approach by which explainable object detection models may be developed for medical imaging.Critical relevance statementWSUnet learns to segment images at voxel level, training only with image-level labels. A Unet backbone first generates a voxel-level probability map and then extracts the maximum voxel prediction as the image-level prediction. Thus, training uses only image-level annotations, reducing human workload. WSUnet’s voxel-level predictions provide a causally verifiable explanation for its image-level prediction, improving interpretability.Key points• Explainability and interpretability are essential for reliable medical image classifiers.• This study applies weakly supervised segmentation to generate explainable image classifiers.• The weakly supervised Unet inherently explains its image-level predictions at voxel level.Graphical

Sensitivity of Survival Analysis Metrics

Survival analysis models allow for predicting the probability of an event over time. The specificity of the survival analysis data includes the distribution of events over time and the proportion of classes. Late events are often rare and do not correspond to the main distribution and strongly affect the quality of the models and quality assessment. In this paper, we identify four cases of excessive sensitivity of survival analysis metrics and propose methods to overcome them. To set the equality of observation impacts, we adjust the weights of events based on target time and censoring indicator. According to the sensitivity of metrics, AUPRC (area under Precision-Recall curve) is best suited for assessing the quality of survival models, and other metrics are used as loss functions. To evaluate the influence of the loss function, the Bagging model uses ones to select the size and hyperparameters of the ensemble. The experimental study included eight real medical datasets. The proposed modifications of IBS (Integrated Brier Score) improved the quality of Bagging compared to the classical loss functions. In addition, in seven out of eight datasets, the Bagging with new loss functions outperforms the existing models of the scikit-survival library.

Using Machine Learning to Predict Bleeding after Cardiac Surgery.

The primary objective was to predict bleeding after cardiac surgery with machine learning using the data from the ANZSCTS (Australia New Zealand Society of Cardiac and Thoracic Surgeons) Cardiac Surgery Database, cardiopulmonary bypass perfusion database, ICU (intensive care unit) database and laboratory results. We obtained surgical, perfusion, ICU and laboratory data from a single Australian tertiary cardiac surgical hospital from February 2015 to March 2022 and included 2000 patients undergoing cardiac surgery. We trained our models to predict either the Papworth definition or Dyke et al.'s universal definition of perioperative bleeding. Our primary outcome was the performance of our machine learning algorithms using sensitivity, specificity, positive and negative predictive values, accuracy, AUROC (area under receiver operating characteristics curve) and AUPRC (area under precision recall curve). Of the 2000 patients undergoing cardiac surgery, 13.3% (226/2000) had bleeding using the Papworth definition and 17.2% (343/2000) had moderate to massive bleeding using Dyke et al.'s definition. The best performing model based on AUPRC was the Ensemble Voting Classifier model for both Papworth (AUPRC 0.310, AUROC 0.738) and Dyke definitions of bleeding (AUPRC 0.452, AUROC 0.797). Machine learning can incorporate routinely collected data from various datasets to predict bleeding after cardiac surgery.

Accurately predicting the risk of unfavorable outcomes after endovascular coil therapy in patients with aneurysmal subarachnoid hemorrhage: an interpretable machine learning model.

Despite endovascular coiling as a valid modality in treatment of aneurysmal subarachnoid hemorrhage (aSAH), there is a risk of poor prognosis. However, the clinical utility of previously proposed early prediction tools remains limited. We aimed to develop a clinically generalizable machine learning (ML) models for accurately predicting unfavorable outcomes in aSAH patients after endovascular coiling. Functional outcomes at 6 months after endovascular coiling were assessed via the modified Rankin Scale (mRS) and unfavorable outcomes were defined as mRS 3-6. Five ML algorithms (logistic regression, random forest, support vector machine, deep neural network, and extreme gradient boosting) were used for model development. The area under precision-recall curve (AUPRC) and receiver operating characteristic curve (AUROC) was used as main indices of model evaluation. SHapley Additive exPlanations (SHAP) method was applied to interpret the best-performing ML model. A total of 371 patients were eventually included into this study, and 85.4% of them had favorable outcomes. Among the five models, the DNN model had a better performance with AUPRC of 0.645 (AUROC of 0.905). Postoperative GCS score, size of aneurysm, and age were the top three powerful predictors. The further analysis of five random cases presented the good interpretability of the DNN model. Interpretable clinical prediction models based on different ML algorithms have been successfully constructed and validated, which would serve as reliable tools in optimizing the treatment decision-making of aSAH. Our DNN model had better performance to predict the unfavorable outcomes at 6 months in aSAH patients compared with Yan's nomogram model.

Precision medicine for traumatic cervical spinal cord injuries: accessible and interpretable machine learning models to predict individualized in-hospital outcomes

BACKGROUND CONTEXTA traumatic spinal cord injury (SCI) can cause temporary or permanent motor and sensory impairment, leading to serious short and long-term consequences that can result in significant morbidity and mortality. The cervical spine is the most commonly affected area, accounting for about 60% of all traumatic SCI cases. PURPOSEThis study aims to employ machine learning (ML) algorithms to predict various outcomes, such as in-hospital mortality, nonhome discharges, extended length of stay (LOS), extended length of intensive care unit stay (ICU-LOS), and major complications in patients diagnosed with cervical SCI (cSCI). STUDY DESIGNOur study was a retrospective machine learning classification study aiming to predict the outcomes of interest, which were binary categorical variables, in patients diagnosed with cSCI. PATIENT SAMPLEThe data for this study were obtained from the American College of Surgeons (ACS) Trauma Quality Program (TQP) database, which was queried to identify patients who suffered from cSCI between 2019 and 2021. OUTCOME MEASURESThe outcomes of interest of our study were in-hospital mortality, nonhome discharges, prolonged LOS, prolonged ICU-LOS, and major complications. The study evaluated the models' performance using both graphical and numerical methods. The receiver operating characteristic (ROC) and precision-recall curves (PRC) were used to assess model performance graphically. Numerical evaluation metrics included AUROC, balanced accuracy, weighted area under PRC (AUPRC), weighted precision, and weighted recall. METHODSThe study employed data from the American College of Surgeons (ACS) Trauma Quality Program (TQP) database to identify patients with cSCI. Four ML algorithms, namely XGBoost, LightGBM, CatBoost, and Random Forest, were utilized to develop predictive models. The most effective models were then incorporated into a publicly available web application designed to forecast the outcomes of interest. RESULTSThere were 71,661 patients included in the analysis for the outcome mortality, 67,331 for the outcome nonhome discharges, 76,782 for the outcome prolonged LOS, 26,615 for the outcome prolonged ICU-LOS, and 72,132 for the outcome major complications. The algorithms exhibited an AUROC value range of 0.78 to 0.839 for in-hospital mortality, 0.806 to 0.815 for nonhome discharges, 0.679 to 0.742 for prolonged LOS, 0.666 to 0.682 for prolonged ICU-LOS, and 0.637 to 0.704 for major complications. An open access web application was developed as part of the study, which can generate predictions for individual patients based on their characteristics. CONCLUSIONSOur study suggests that ML models can be valuable in assessing risk for patients with cervical cSCI and may have considerable potential for predicting outcomes during hospitalization. ML models demonstrated good predictive ability for in-hospital mortality and nonhome discharges, fair predictive ability for prolonged LOS, but poor predictive ability for prolonged ICU-LOS and major complications. Along with these promising results, the development of a user-friendly web application that facilitates the integration of these models into clinical practice is a significant contribution of this study. The product of this study may have significant implications in clinical settings to personalize care, anticipate outcomes, facilitate shared decision making and informed consent processes for cSCI patients.

Development and use of machine learning models for prediction of male sling success: A proof-of-concept institutional evaluation.

For mild to moderate male stress urinary incontinence (SUI), transobturator male slings remain an effective option for management. We aim to use a machine learning (ML)-based model to predict those who will have a long-term success in managing SUI with male sling. All transobturator male sling cases from August 2006 to June 2012 by a single surgeon were reviewed. Outcome of interest was defined as 'cure': complete dryness with 0 pads used, without the need for additional procedures. Clinical variables included in ML models were: number of pads used daily, age, height, weight, race, incontinence type, etiology of incontinence, history of radiation, smoking, bladder neck contracture, and prostatectomy. Model performance was assessed using AUROC, AUPRC, and F1-score. A total of 181 patients were included in the model. The mean followup was 56.4 months (standard deviation [SD] 41.6). Slightly more than half (53.6%, 97/181) of patients had procedural success. Logistic regression, K-nearest neighbor (KNN), naive Bayes, decision tree, and random forest models were developed using ML. KNN model had the best performance, with AUROC of 0.759, AUPRC of 0.916, and F1-score of 0.833. Following ensemble learning with bagging and calibration, KNN model was further improved, with AUROC of 0.821, AUPRC of 0.921, and F-1 score of 0.848. ML-based prediction of long-term transobturator male sling is feasible. The low numbers of patients used to develop the model prompt further validation and development of the model but may serve as a decision-making aid for practitioners in the future.

Predicting sepsis onset using a machine learned causal probabilistic network algorithm based on electronic health records data

Sepsis is a leading cause of mortality and early identification improves survival. With increasing digitalization of health care data automated sepsis prediction models hold promise to aid in prompt recognition. Most previous studies have focused on the intensive care unit (ICU) setting. Yet only a small proportion of sepsis develops in the ICU and there is an apparent clinical benefit to identify patients earlier in the disease trajectory. In this cohort of 82,852 hospital admissions and 8038 sepsis episodes classified according to the Sepsis-3 criteria, we demonstrate that a machine learned score can predict sepsis onset within 48 h using sparse routine electronic health record data outside the ICU. Our score was based on a causal probabilistic network model—SepsisFinder—which has similarities with clinical reasoning. A prediction was generated hourly on all admissions, providing a new variable was registered. Compared to the National Early Warning Score (NEWS2), which is an established method to identify sepsis, the SepsisFinder triggered earlier and had a higher area under receiver operating characteristic curve (AUROC) (0.950 vs. 0.872), as well as area under precision-recall curve (APR) (0.189 vs. 0.149). A machine learning comparator based on a gradient-boosting decision tree model had similar AUROC (0.949) and higher APR (0.239) than SepsisFinder but triggered later than both NEWS2 and SepsisFinder. The precision of SepsisFinder increased if screening was restricted to the earlier admission period and in episodes with bloodstream infection. Furthermore, the SepsisFinder signaled median 5.5 h prior to antibiotic administration. Identifying a high-risk population with this method could be used to tailor clinical interventions and improve patient care.

Root Dilaceration Using Deep Learning: A Diagnostic Approach

Understanding usual anatomical structures and unusual root formations is crucial for root canal treatment and surgical treatments. Root dilaceration is a tooth formation with sharp bends or curves, which causes dental treatments to fail, especially root canal treatments. The aim of the study was to apply recent deep learning models to develop an artificial intelligence-based computer-aided detection system for root dilaceration in panoramic radiographs. A total of 983 objects in 636 anonymized panoramic radiographs were initially labelled by an oral and maxillofacial radiologist and were then used to detect root dilacerations. A total of 19 state-of-the-art deep learning models with distinct backbones or feature extractors were used with the integration of alternative frameworks. Evaluation was carried out using Common Objects in Context (COCO) detection evaluation metrics, mean average precision (mAP), accuracy, precision, recall, F1 score and area under precision-recall curve (AUC). The duration of training was also noted for each model. Considering the detection performance of all models, mAP, accuracy, precision, recall, and F1 scores of up to 0.92, 0.72, 0.91, 0.87 and 0.83, respectively, were obtained. AUC were also analyzed to better understand where errors originated. It was seen that background confusion limited performance. The proposed system can facilitate root dilaceration assessment and alleviate the burden of clinicians, especially for endodontists and surgeons.

DA-SRN: Omics data analysis based on the sample network optimization for complex diseases

Effective biomarker identification and accurate sample label prediction are still challenging for complex diseases. Patient similarity network (PSN) analysis is a powerful tool in disease omics data analysis. The topology of PSN can reflect the discriminative ability of the corresponding feature space on which the sample network is built. In this study, a novel omics data analysis method based on the sample reference network (DA-SRN) is proposed to identify the potential biomarkers and predict the sample categories. DA-SRN defines the informative features and the sample reference network in optimizing the network structure by genetic algorithm. It labels the samples based on the graph neural network, the reference network and the selected informative features. DA-SRN was compared with nine efficient omics data analysis methods on the genomics, metabolomics and transcriptomics datasets to show its validation. The comparison results showed that it outperformed the other methods in area under receiver operating characteristic curve (AUROC), sensitivity, specificity and area under precision-recall curve (AUPRC) in most cases. Besides, the important metabolites identified by DA-SRN for the type 2 diabetes (T2D) metabolomics data were further examined. The pathway analysis revealed the close relationships between the identified metabolites and the critical metabolic pathways related to the occurrence and development of T2D. The experimental results illustrate that DA-SRN can extract the valuable information from the complex omics data by analyzing the sample relationship, and is promising in biomarker identification and sample discrimination for complex diseases.

Wearable movement-tracking data identify Parkinson's disease years before clinical diagnosis.

Parkinson's disease is a progressive neurodegenerative movement disorder with a long latent phase and currently no disease-modifying treatments. Reliable predictive biomarkers that could transform efforts to develop neuroprotective treatments remain to be identified. Using UK Biobank, we investigated the predictive value of accelerometry in identifying prodromal Parkinson's disease in the general population and compared this digital biomarker with models based on genetics, lifestyle, blood biochemistry or prodromal symptoms data. Machine learning models trained using accelerometry data achieved better test performance in distinguishing both clinically diagnosed Parkinson's disease (n = 153) (area under precision recall curve (AUPRC) 0.14 ± 0.04) and prodromal Parkinson's disease (n = 113) up to 7 years pre-diagnosis (AUPRC 0.07 ± 0.03) from the general population (n = 33,009) compared with all other modalities tested (genetics: AUPRC = 0.01 ± 0.00, P = 2.2 × 10-3; lifestyle: AUPRC = 0.03 ± 0.04, P = 2.5 × 10-3; blood biochemistry: AUPRC = 0.01 ± 0.00, P = 4.1 × 10-3; prodromal signs: AUPRC = 0.01 ± 0.00, P = 3.6 × 10-3). Accelerometry is a potentially important, low-cost screening tool for determining people at risk of developing Parkinson's disease and identifying participants for clinical trials of neuroprotective treatments.

Prediction of multiple neuropathologic changes from features available during life

AbstractBackgroundNeuropathologic changes are central for both understanding of the patients’ brains and making the definitive diagnosis of dementia‐related diseases. However, many of them are only obtainable post‐mortem. To make this information available while an individual is still alive, this study developed machine learning models that predict neuropathologic changes based on features obtainable during life.MethodThe multi‐site post‐mortem data (n∼5000) was obtained from National Alzheimer’s Coordinating Center. A multitask long‐short term memory‐based neural network architecture was developed with custom loss to predict the 13 neuropathologic changes from features during life measured longitudinally. The performance of the model was evaluated using entire unseen sites as test sets. Evaluation metrics include area under receiver’s operating curve (AUROC) and area under precision recall curve (AUPRC).ResultThe model was able to predict Alzheimer’s Disease neuropathologic changes (ADNC) and any Alzheimer’s‐related pathologies, such as Braak score, with great sensitivity, specificity, and precision (for example AUROC = 0.85; AUPRC = 0.96 for ADNC). Apart from these, the model can also predict hippocampal sclerosis accurately (AUROC = 0.79; AUPRC = 0.80) and Lewy Body disease at higher precision than clinician’s diagnosis. Model interpretation shows patterns in neuropsychological tests that are predictive of pathologic changes. Additionally, model error analysis revealed factors, such as resilient case ratios, that explain variation in performance between sites.ConclusionPatterns of measurable features during life can be used by machine learning to predict ADNC, hippocampal sclerosis, and to lesser extent Lewy Body.

Machine learning and postoperative complications: Determining risk before surgery in patients with cancer.

6575 Background: In patients with advanced cancer who need unplanned surgery, shared decision making is complex. Surgical risk assessment with high accuracy and in real time may help improve prediction of postoperative outcomes. We previously trained and tested an explainable machine learning model (MLM) to estimate postoperative Clavien-Dindo grade 3 or higher (CD 3+) complication risk in cancer inpatients undergoing unplanned surgery. In this prospective validation study, we report the performance of the MLM Surgical Risk Score (SRS) in the prediction of postoperative outcomes as well as determine association of the risk score with hospital length of stay and mortality. Methods: A MLM SRS for cancer inpatients requiring an unplanned, same-hospitalization major surgery was generated using real-time patient specific EHR data. The SRS was integrated into surgeon preoperative workflow. Postoperative outcomes including complications, length of hospital stay, and 30-day mortality were reported. The SRS was dichotomized as high or low based on prior validation threshold for postoperative CD3+ complication development. Model performance was assessed with area under receiver operating characteristic (AUROC) and area under precision-recall curves (AUPRC). Differences in outcomes between high and low-risk SRS groups were studied using Mann-Whitney U test and Fisher's exact test. Results: During the study period, 135 cancer inpatients were assessed by surgeons for 166 major unplanned operations. Of the 166 operations, 126 were completed and 40 were cancelled. The SRS was high in 81 (49%) and low in 85 (51%). The most common cancer diagnosis was colorectal (n=39, 28.8%). Operations performed with the highest frequencies were for enteral access (e.g.gastrostomy) followed by exploratory laparotomy and enterectomy. CD 3+ complications were identified in 31% of the completed operations (n=39/126) and included death (n = 10); abscess (n=4); postoperative hemorrhage (n=3); and anastomotic leak (n=2). The SRS predicted CD 3+ complications with a precision of47% and recall of 73%, resulting in an AUROC of 0.75 and AUPRC of 0.58. When comparing patients with high-risk to low-risk SRS, median overall LOS was 10 vs 7 days (p=0.013), median LOS after operation was 9 vs 6 days (p=0.003), 30-day mortality was 19.0% vs 5.3% (p=0.015), and 30-day postoperative mortality was 13.2% vs 3.3% (p=0.04). Conclusions: In this prospective study, preoperative SRS generated from a trained MLM on real-time EHR data demonstrated an excellent ability to predict individual risk of postoperative complications. In addition, patients with high SRS have longer postoperative LOS and increased mortality. By improving discrimination of risk of postoperative complications and informing on treatment outcomes, integration of the MLM SRS into the consenting process may enable better informed decision making in cancer inpatients.

Enabling personalized perioperative risk prediction by using a machine-learning model based on preoperative data

Preoperative risk assessment is essential for shared decision-making and adequate perioperative care. Common scores provide limited predictive quality and lack personalized information. The aim of this study was to create an interpretable machine-learning-based model to assess the patient’s individual risk of postoperative mortality based on preoperative data to allow analysis of personal risk factors. After ethical approval, a model for prediction of postoperative in-hospital mortality based on preoperative data of 66,846 patients undergoing elective non-cardiac surgery between June 2014 and March 2020 was created with extreme gradient boosting. Model performance and the most relevant parameters were shown using receiver operating characteristic (ROC−) and precision-recall (PR-) curves and importance plots. Individual risks of index patients were presented in waterfall diagrams. The model included 201 features and showed good predictive abilities with an area under receiver operating characteristic (AUROC) curve of 0.95 and an area under precision-recall curve (AUPRC) of 0.109. The feature with the highest information gain was the preoperative order for red packed cell concentrates followed by age and c-reactive protein. Individual risk factors could be identified on patient level. We created a highly accurate and interpretable machine learning model to preoperatively predict the risk of postoperative in-hospital mortality. The algorithm can be used to identify factors susceptible to preoperative optimization measures and to identify risk factors influencing individual patient risk.

Deep-GA-Net for Accurate and Explainable Detection of Geographic Atrophy on OCT Scans

To propose Deep-GA-Net, a 3-dimensional (3D) deep learning network with 3D attention layer, for the detection of geographic atrophy (GA) on spectral domain OCT (SD-OCT) scans, explain its decision making, and compare it with existing methods. Deep learning model development. Three hundred eleven participants from the Age-Related Eye Disease Study 2 Ancillary SD-OCT Study. A dataset of 1284 SD-OCT scans from 311 participants was used to develop Deep-GA-Net. Cross-validation was used to evaluate Deep-GA-Net, where each testing set contained no participant from the corresponding training set. En face heatmaps and important regions at the B-scan level were used to visualize theoutputs of Deep-GA-Net, and 3 ophthalmologists graded the presence or absence of GA in them to assess the explainability (i.e., understandability and interpretability) of its detections. Accuracy, area under receiver operating characteristic curve (AUC), area under precision-recall curve (APR). Compared with other networks, Deep-GA-Net achieved the best metrics, with accuracy of 0.93, AUC of 0.94, and APR of 0.91, and received the best gradings of 0.98 and 0.68 on the en face heatmap and B-scan grading tasks, respectively. Deep-GA-Net was able to detect GA accurately from SD-OCT scans. The visualizations of Deep-GA-Net were more explainable, as suggested by 3 ophthalmologists. The code and pretrained models are publicly available at https://github.com/ncbi/Deep-GA-Net. The author(s) have no proprietary or commercial interest in any materials discussed in this article.