Hold-out Set Research Articles

Abstract 4125281: Detection of Acute Myocardial Infarction and Ischemia From Lead-I ECG Using Deep Learning

Introduction: While smartwatches that acquire Lead-I electrocardiogram (ECG) have shown promise in detecting arrhythmias, the detection of ischemic events using Lead-I alone remains understudied. We aim to build and evaluate a deep-learning model to determine whether Lead-I ECG can identify ischemia/infarction in a 12-lead ECG. Methods: We developed a deep-learning model that can identify MI/ischemia using only Lead-I ECG signal as input. For this task, we utilized PTB-XL, a large publicly available dataset of 12-Lead clinical ECG with associated diagnostic labels. Our experiment utilized 8781 ECGs whose labels were validated by at-least one cardiologist with 100% confidence scores. This dataset includes 1391 ECGs that are labeled as either ischemic ST/T changes or MI, and 7390 ECGs with either “normal” labels or non-ischemic ST/T abnormalities. We split the dataset in training, validation, and holdout sets (80-10-10) with equal stratification of the diagnosis classes. The holdout test set contains 909 non-ischemic ECGs and 233 ECGs with MI/ischemia diagnosis, 106 of which are labeled as acute MI or ischemic ST/T changes and 127 labeled as other MI chronicity. The discriminatory ability of the model is evaluated with the Area Under the receiver-operating Curve (AUC), sensitivity, specificity, and the predictive values. We further analyze the model’s capability in discriminating the cohort of acute MI and ischemia diagnoses from other MI as well as from the non-ischemic cohort. Results: The model has an AUC of 0.88 for discriminating an MI/ischemia vs normal. Using a threshold corresponding to 80% sensitivity, the corresponding specificity is 79% and the negative and positive predictive values are 94% and 50% (Table-1). Although the model was trained to identify the combined class of MI/ischemia, it can differentiate acute MI and ischemia cohorts from other MI as well as from non-ischemic diagnoses with p-values<10 -4 (Table-2). Lastly, with the same threshold, the sensitivity for detecting acute inferior myocardial infarctions or inferior ischemia – a class that is difficult to detect with Lead-I alone – is 82%. Conclusions: Our deep learning model, which was trained on Lead-I ECG alone, can effectively detect acute MI and ischemia that would have been identified using a 12-lead ECG. This method presents a proof-of-concept for applications in home settings. Further prospective validation using wearable ECG monitors that record lead-I is warranted.

Abstract 4138373: Machine learning identifies clinically distinct phenotypes in patients with aortic regurgitation.

Background: Aortic regurgitation (AR) is a prevalent valve disease with a long latent period to symptoms. Recent data has suggested the role of novel markers of myocardial overload in assessing onset of decompensation. Method: We sought to evaluate the role of unsupervised cluster analyses in identifying different clinical clusters, including clinical status, and a large number of echocardiographic variables including left ventricular (LV) volumes, and their association with mortality. Patients with ≥moderate-severe chronic AR identified using echocardiography at Mayo Clinic, Rochester were retrospectively analyzed. Primary outcome was all-cause mortality censored at aortic valve surgery/last follow-up. Uniform Manifold Approximation and Projection (UMAP) with K -means algorithm was used to cluster patients using clinical and, echocardiographic variables at the time of presentation. Missing data were imputed with the Multiple Imputation by Chained Equations (MICE) method. A supervised approach trained on the training set was used to find cluster membership in a hold-out validation set. Log-rank tests were used to assess differences in mortality rates between the clusters, both in the training and validation sets. Results: Three distinct clusters were identified among 1100 patients (log-rank P for survival <0.001). Cluster 1 (n=337), which included younger males with severe AR but fewer symptoms, showed the best survival, 75.6% (69.5, 82.3). Cluster 2 (n=235), older and more females with elevated filling pressures, showed intermediate survival of 64.2 % (56.8, 72.5). Cluster 3 (n=253), characterized by severe symptomatic AR, demonstrated the lowest survival of 45.3 % (34.4, 59.8) at 5 years. Similar clusters were formed in the internal validation cohort. Conclusion: Distinct clusters with variable echocardiographic features and mortality differences exist within patients with chronic ≥moderate-severe AR. Recognizing these clusters can refine individual risk stratification and clinical decision-making after verification in future prospective studies.

NIMG-66. DEFINING PROGRESSIVE DISEASE: LOCALIZING VOXEL-AT-RISK ON MAGNETIC RESONANCE IMAGING USING DEEP LEARNING

Abstract Follow-up magnetic resonance imaging (MRI) of post-treatment glioblastoma (GBM) can be diagnostically challenging in differentiating progressive disease (PD) from coexisting treatment-related changes. The study aim was to train a machine learning model using multiple MRI timepoints from treated GBM patients with subsequent PD or at least six months of tumor-like, persistently stable disease (SD) to estimate PD risk. A retrospective, post-treatment GBM cohort (n=189) with 453 timepoints was identified with available MRI sequences (T1-weighted with and without contrast, T2-weighted, and fluid-attenuated inversion recovery). PD cases (n=154) showed new, enhancing PD followed by death within 3-6 months; timepoints included the MRI on the PD diagnosis date and the one directly prior. SD cases (n=35) exhibited stable, tumor-like enhancement over timepoints. Cases underwent voxel-wise annotation of PD versus SD. A multiclass, three-dimensional nnU-Net was trained to label voxels as PD, SD, or neither using timepoint sequences with five-fold cross-validation. Model performance was assessed using the ratio of predicted PD/SD voxels compared to each case’s actual PD or SD status. Classification accuracy of PD versus SD status, PD case-wise sensitivity and specificity, and a Receiver Operating Characteristic (ROC) analysis using the PD/SD ratio were presented. The five-fold model ensemble was evaluated on a holdout test set. Validation set performance showed 74.3% accuracy, 0.73 PD case-wise sensitivity, and 0.80 specificity. The holdout set exhibited 79.8% accuracy, 0.81 sensitivity, and 0.78 specificity for PD cases. The ROC analysis showed an area under the curve of 0.80 and 0.83 on the validation and holdout sets, respectively. The model showed high performance and classification accuracy in predicting which voxels were at risk of PD in post-treatment GBM cases. The model also differentiated between PD and SD regions. The model’s predicted PD/SD ratio may be clinically used to stratify patients’ PD risk.

NIMG-75. SIMILARITY DISTANCES-BASED DELTA RADIOMICS (SDDR) ENABLES LONGITUDINAL MRI COMPARISONS FOR SURVIVAL PREDICTION IN GLIOMAS: A MULTI-INSTITUTIONAL VALIDATION

Abstract Radiomics (computerized feature analysis) on treatment-naive MRI scans has demonstrated great value in outcome prediction for Glioblastoma (GBM). However, delta radiomics (analysis of radiomic feature variation between different events, e.g., before and after treatment) has not been explored on account of challenges with precise spatial correspondences between pre- and post-operative scans. This work presents a survival prediction model for GBM, Similarity Distances-based delta radiomics (SDDR), based on the hypothesis that similarity distance-measures between radiomic features on pre-, and post-treatment images can accurately capture temporal changes within-and-around the tumor, and provide improved prediction of overall survival, compared to models based on a single timepoint. 150 patients from 2 publicly-available Glioma repositories (University of California San Francisco “UCSF” (n=48), LUMIERE (n=67)) and 2 two proprietary datasets (from xCures (n=13), and Cleveland Clinic (CCF) (n=74)) were obtained. UCSF and LUMIERE cohorts were used as separate hold-out sets, in each of which the remaining collections were used as discovery set. Following pre-processing, tumor segmentation into Edema (ED) and Enhancing tumor (ET) were performed. SDDR was then computed, which is the distance between probability densities functions from 156 gray co-occurrence (GLCM) and 52 gradient-based co-occurrences statistics (COLLAGE) features extracted from pre and-post-treatment MRI. Six statistical distances (e.g., Earth mover distance) normalized by the scanning interval (in weeks) yielded n=2496 SDDR measurements per tumor region (1248/region, 936 delta-GLCM, and 312 delta-COLLAGE). The same set of features were computed from pretreatment MRI (Pre-R) for comparison. Features were fed to LASSO-Cox regression for survival analysis. SDDR for GLCM features extracted from ET outperformed Pre-R, and demonstrated significant differences between high and low risk groups for both hold out tests: UCSF (p=0.0069, HR =2.8, C-index=0.69) and LUMIERE (p=0.0061, HR = 1.9, C-index 0.65). SDDR-based multi-institutional survival analysis demonstrated promise in reliable longitudinal risk-stratification in GBM.

Authenticity markers of cultivated blueberries (Vaccinium sp.) and European wild blueberry (V. myrtillus) assessed by hyperspectral imaging and machine learning algorithms

The present study was undertaken to tackle food fraud in the food chain by developing a novel panel of analytical fingerprints to distinguish the fruit of bilberry (Vaccinium myrtillus), a wild European blueberry grown in Finland, from their commercial cultivated counterparts based on V. corymbosum or other originally American Vaccinium species. The analytical fingerprints were derived using hyperspectral imaging (HSI). The cultivated domestic blueberry samples from Finland (N = 188), those imported (N = 338), and domestic Finnish wild bilberry fruit (N = 247) were imaged using Specim IQ and FX17 HSI cameras. The fruit HSI data were then classified using seven machine learning model types. The supervised evolutionary feature selection algorithm was used to recognize the most separating subset of wavelengths (nm predictors, colors) between the fruit class labels, which were 1) wild bilberry fruit vs. cultivated blueberry fruit and 2) domestic cultivated blueberry fruit vs. imported cultivated blueberry fruit. Moreover, we ran 190,822 individual models and 54,137 predictor combinations to determine whether subsets or all wavelengths better distinguished fruit class labels. Overall, the best model type, the rbf kernel support vector machine (rbf-SVM), showed excellent prediction performance when classifying wild bilberries vs. cultivated blueberries with HSI data obtained using both camera types (4.1% and 3.2% holdout set errors). Also the rbf-SVM performed well when classifying domestic cultivated blueberries vs. those imported with HSI data obtained using an IQ camera (2.3% holdout set error). Our results showed that hyperspectral imaging could predict food fraud attempts concerning the Vaccinium fruit of different origin and species.

Using intrahost single nucleotide variant data to predict SARS-CoV-2 detection cycle threshold values.

Over the last four years, each successive wave of the COVID-19 pandemic has been caused by variants with mutations that improve the transmissibility of the virus. Despite this, we still lack tools for predicting clinically important features of the virus. In this study, we show that it is possible to predict the PCR cycle threshold (Ct) values from clinical detection assays using sequence data. Ct values often correspond with patient viral load and the epidemiological trajectory of the pandemic. Using a collection of 36,335 high quality genomes, we built models from SARS-CoV-2 intrahost single nucleotide variant (iSNV) data, computing XGBoost models from the frequencies of A, T, G, C, insertions, and deletions at each position relative to the Wuhan-Hu-1 reference genome. Our best model had an R2 of 0.604 [0.593-0.616, 95% confidence interval] and a Root Mean Square Error (RMSE) of 5.247 [5.156-5.337], demonstrating modest predictive power. Overall, we show that the results are stable relative to an external holdout set of genomes selected from SRA and are robust to patient status and the detection instruments that were used. This study highlights the importance of developing modeling strategies that can be applied to publicly available genome sequence data for use in disease prevention and control.

Comprehensive machine learning models for prediction of heart failure in 476,393 women and men from the UK Biobank reveal sex differences and underutilized risk factors

Abstract Background Machine learning (ML) models provide potential advantage over ‘traditional’ regression models in heart failure (HF) prediction. Objective To compare performances of Cox PH models and ML survival models for incident HF in men and women without prevalent ischemic heart disease (IHD). We also aimed to identify potential high-risk precursors otherwise ignored by conventional survival models, and to investigate differences between sex-specific models. Methods We included 476,393 participants (55.6% women) from the UK Biobank, after excluding participants with a history of HF or IHD, and defined sex-specific datasets. We predicted incident HF events using over 400 baseline characteristics. We constructed multivariable Cox PH models, which included all predictor variables and subsequently only those remaining after LASSO stability selection. We also developed two supervised ML models (Random Survival Forest (RSF), eXtreme Gradient Survival Boosting (XGBoost)). We identified the 15 most important sex-specific predictors in each model and performances were compared using the C-index. Models were validated using hold-out sets. Results During 12.3 ± 1.9 years of follow-up, 4680 (1.76%) women and 6631 (3.14%) men developed incident HF. XGBoost showed the best performance during model training (C-index, training: 0.89 in men, 0.97 in women; validation 0.77 in men, 0.80 in women). The multivariable Cox model performed second-best (C-index, training: 0.78 in men, 0.82 in women; validation: 0.76 in men, 0.78 in women). RSF performed slightly worse (C-index, training: 0.75 in men, 0.79 in women; validation: 0.75 in men, 0.79 in women) but did not show performance drop during validation. LASSO stability selection performed similar to RSF. Age, self-reported lifetime treatments and medications, cystatin-C, waist circumference and FEV1-scores were identified as strong risk factors in all models for both sexes. Reduced albumin levels and elevated HbA1c were more strongly associated with high risk in men, while elevated systolic BP showed higher importance in women. Traditional Cox models observed CRP as important only in men, while the ML models identified CRP as important for both sexes. Neutrophil count was considered a strong risk factor in both sexes in the traditional Cox models, yet it was not among the most important predictors in both ML models. Presence of other heart disease (which included a.o. pericardial disease, valve disorders and arrhythmias) was an important predictor variable only in the ML models. Conclusion ML models showed similar performance to Cox PH models for HF prediction. Despite this, differences in predictor importance were identified between models. Sex-specific risk predictors were found, and FEV1 score, which is not commonly included in existing models, was identified as an important risk factor. These results suggest that ML models may reveal additional insights that would otherwise remain unnoticed.

Towards the prediction of drug solubility in binary solvent mixtures at various temperatures using machine learning

Drug solubility is an important parameter in the drug development process, yet it is often tedious and challenging to measure, especially for expensive drugs or those available in small quantities. To alleviate these challenges, machine learning (ML) has been applied to predict drug solubility as an alternative approach. However, the majority of existing ML research has focused on the predictions of aqueous solubility and/or solubility at specific temperatures, which restricts the model applicability in pharmaceutical development. To bridge this gap, we compiled a dataset of 27,000 solubility datapoints, including solubility of small molecules measured in a range of binary solvent mixtures under various temperatures. Next, a panel of ML models were trained on this dataset with their hyperparameters tuned using Bayesian optimization. The resulting top-performing models, both gradient boosted decision trees (light gradient boosting machine and extreme gradient boosting), achieved mean absolute errors (MAE) of 0.33 for LogS (S in g/100 g) on the holdout set. These models were further validated through a prospective study, wherein the solubility of four drug molecules were predicted by the models and then validated with in-house solubility experiments. This prospective study demonstrated that the models accurately predicted the solubility of solutes in specific binary solvent mixtures under different temperatures, especially for drugs whose features closely align within the solutes in the dataset (MAE < 0.5 for LogS). To support future research and facilitate advancements in the field, we have made the dataset and code openly available.Scientific contributionOur research advances the state-of-the-art in predicting solubility for small molecules by leveraging ML and a uniquely comprehensive dataset. Unlike existing ML studies that predominantly focus on solubility in aqueous solvents at fixed temperatures, our work enables prediction of drug solubility in a variety of binary solvent mixtures over a broad temperature range, providing practical insights on the modeling of solubility for realistic pharmaceutical applications. These advancements along with the open access dataset and code support significant steps in the drug development process including new molecule discovery, drug analysis and formulation.Graphical

An artificial intelligence-based landmark model for early detection of aortic stenosis in Cardiac MRI

Abstract Background The utilisation of ratiometric intensity measures from cardiac magnetic resonance imaging (CMR) offers a promising biomarker for the diagnosis of aortic stenosis (AS)[1]. The Ao:LV ratio, an approximation of AS severity, is determined by comparing the relative blood signal intensity in the aorta and left ventricle (LV) within three-chamber (3Ch) cine sequences[1]. In patients with AS, the blood signal intensity of aortic blood above the valve is diminished compared to that within the LV in steady-state free precession (SSFP) 3Ch CMR cine. Although the Ao:LV ratio serves as a reliable indicator of AS severity, manual computation of this ratio may encounter difficulties due to visual obstructions caused by small LV cavities or prominent papillary muscles. Purpose This study aims to leverage artificial intelligence (AI) to automate the Ao:LV ratio analysis from 3Ch SSFP cine images and introduce the Ao:LA (aorta to left atrium) ratio, thus enhancing the diagnostic accuracy for AS severity. Methods Our methodology extends the ratiometric analysis to include both Ao:LV and Ao:LA ratios in three main steps: first, developing an AI-based algorithm for automated detection and tracking of cardiac landmarks; second, using these landmarks to automatically compute regions of interest (ROIs) in the cardiac chambers; and third, training and validating classification models with 5-fold cross-validation to select the most effective for AS severity classification. The final model's performance is assessed on a holdout test set. Data for model training and evaluation were sourced from CMR scans across three hospitals and two scanner types. Results We analysed 220 patients of which 78 patients had no AS and 122 had AS (mild AS, n = 29; moderate AS; n = 35, severe AS; n = 78). Our model yielded high efficacy in classifying AS, particularly in differentiating any grade of AS from a normal classification, which is clinically relevant. The Ao:LA ratio demonstrated a stronger correlation with AS severity class than Ao:LV, suggesting a more reliable biomarker. The proposed model demonstrated robust performance in AS classification, achieving an Area Under the Curve (AUC) of 0.845 in the automated binary classification of AS at any grade. The model exhibited a precision of 0.889, recall of 0.865, and an F1 score of 0.877 on a holdout set. Conclusion By the integration of AI in automating the analysis of CMR images through a novel ratiometric approach that includes Ao:LV and Ao:LA ratios, our model significantly improves the accuracy of early AS detection. This advancement enhances imaging strategies for AS evaluation, promoting more efficient and effective use of cardiac MRI sequences as a diagnostic tool for aortic stenosis. It has the potential to be implemented in real-time scanning protocols as a clinical decision support system.AI-inferred cardiac landmarks.ROC curve delineating AS prediction.

Serum metabolite and metal ions profiles for breast cancer screening

Enhancing early-stage breast cancer detection requires integrating additional screening methods with current diagnostic imaging. Omics screening, using easily collectible serum samples, could serve as an initial step. Alongside biomarker identification capabilities, omics analysis allows for a comprehensive analysis of prevalent histological types—DCIS and IDC. Employing metabolomics, metallomics, and machine learning, could yield accurate screening models with valuable insights into organism responses. Serum samples of confirmed breast cancer patients were utilized to analyze metabolite and metal ion profiles, using two distinct analysis methods, proton NMR for metabolomics and ICP-OES for metallomics. The resulting responses were then subjected to discriminant analysis, progression biomarker exploration, examination of correlations between patients’ metabolites and metal ions, and the impact of age and menopause status. Measured NMR spectra and metabolite relative integrals were used to achieve statistically significant discrimination through MVA between breast cancer and control groups. The analysis identified 24 metabolites and 4 metal ions crucial for discrimination. Furthermore, four metabolites were associated with disease progression. Additionally, there were important correlations and relationships between metabolite relative integrals, metal ion concentrations, and age/menopausal status subgroups. Quantified relative integrals allowed for discrimination between studied subgroups, validated with a holdout set. Feature importance and statistical analysis for metabolomics and metallomics extracted a set of common entities which in combination provides valuable insights into ongoing molecular disturbances and disease progression.

PRECISE-GBM: A RADIOGENOMIC MACHINE LEARNING BASED STUDY TO IDENTIFY PREDICTIVE RADIOMICS FOR EVALUATION OF CANCER IMMUNE SIGNATURE IN IDHW GLIOBLASTOMA

Abstract AIMS The aim of the study is to identify machine learning model based on radiological biomarkers that can predict immune status within tumor microenvironment of IDH wildtype glioblastoma. METHOD This is a retrospective multicenter machine learning based study utilizing open access anonymized matched radiogenomic data of TCGA-GBM, CPTAC, Ivy GAP and REMBRANDT datasets. REMBRANDT data acted as hold out dataset. Imaging data consisted of MRI based radiomics features extracted from deep learning based segmented tumors and transcriptomic (RNA seq/microarray) data consisted of immune scores extracted using CIBERSORTx. Outliers were identified and filtered using principal component analysis. Combat and neuroCombat were utilized for data harmonization for each data type respectively. Radiomic feature selection was performed using LASSO technique with cross validation. Support vector machine (SVM) was trained and tested with 80:20 radiomics data split and a median based binary label of immune score (low and high). Validation of the trained model was performed with hold out dataset. Data query and analysis was performed using python and R statistical software. RESULTS One hundred one patients were included in the study with 15 being on the hold out set. Trained SVM model predicted the immune score label with AUC of 0.79 and balanced accuracy of 0.76 (recall 0.75, F1 score 0.75, precision 0.75) in the test data. In the holdout set, the model predicted the immune score with AUC of 0.70 with specificity of 75% in predicting the cold immune cases. The radiomic signature that acted as the label for this model were a group of first order and second order radiomic features. CONCLUSION Radiogenomic SVM model non-invasively predicts immune status in wildtype glioblastoma microenvironment with good accuracy. This model has potential to be utilized in a prospective glioblastoma clinical trial for immunotherapy.

Ethical considerations of use of hold-out sets in clinical prediction model management

AbstractClinical prediction models are statistical or machine learning models used to quantify the risk of a certain health outcome using patient data. These can then inform potential interventions on patients, causing an effect called performative prediction: predictions inform interventions which influence the outcome they were trying to predict, leading to a potential underestimation of risk in some patients if a model is updated on this data. One suggested resolution to this is the use of hold-out sets, in which a set of patients do not receive model derived risk scores, such that a model can be safely retrained. We present an overview of clinical and research ethics regarding potential implementation of hold-out sets for clinical prediction models in health settings. We focus on the ethical principles of beneficence, non-maleficence, autonomy and justice. We also discuss informed consent, clinical equipoise, and truth-telling. We present illustrative cases of potential hold-out set implementations and discuss statistical issues arising from different hold-out set sampling methods. We also discuss differences between hold-out sets and randomised control trials, in terms of ethics and statistical issues. Finally, we give practical recommendations for researchers interested in the use hold-out sets for clinical prediction models.

Development and testing of a deep learning algorithm to detect lung consolidation among children with pneumonia using hand-held ultrasound.

Severe pneumonia is the leading cause of death among young children worldwide, disproportionately impacting children who lack access to advanced diagnostic imaging. Here our objectives were to develop and test the accuracy of an artificial intelligence algorithm for detecting features of pulmonary consolidation on point-of-care lung ultrasounds among hospitalized children. This was a prospective, multicenter center study conducted at academic Emergency Department and Pediatric inpatient or intensive care units between 2018-2020. Pediatric participants from 18 months to 17 years old with suspicion of lower respiratory tract infection were enrolled. Bedside lung ultrasounds were performed using a Philips handheld Lumify C5-2 transducer and standardized protocol to collect video loops from twelve lung zones, and lung features at both the video and frame levels annotated. Data from both affected and unaffected lung fields were split at the participant level into training, tuning, and holdout sets used to train, tune hyperparameters, and test an algorithm for detection of consolidation features. Data collected from adults with lower respiratory tract disease were added to enrich the training set. Algorithm performance at the video level to detect consolidation on lung ultrasound was determined using reference standard diagnosis of positive or negative pneumonia derived from clinical data. Data from 107 pediatric participants yielded 117 unique exams and contributed 604 positive and 589 negative videos for consolidation that were utilized for the algorithm development process. Overall accuracy for the model for identification and localization of consolidation was 88.5%, with sensitivity 88%, specificity 89%, positive predictive value 89%, and negative predictive value 87%. Our algorithm demonstrated high accuracy for identification of consolidation features on pediatric chest ultrasound in children with pneumonia. Automated diagnostic support on an ultraportable point-of-care device has important implications for global health, particularly in austere settings.

A radiomics nomogram based on multiparametric MRI for diagnosing focal cortical dysplasia and initially identifying laterality

BackgroundFocal cortical dysplasia (FCD) is the most common epileptogenic developmental malformation. The diagnosis of FCD is challenging. We generated a radiomics nomogram based on multiparametric magnetic resonance imaging (MRI) to diagnose FCD and identify laterality early.MethodsForty-three patients treated between July 2017 and May 2022 with histopathologically confirmed FCD were retrospectively enrolled. The contralateral unaffected hemispheres were included as the control group. Therefore, 86 ROIs were finally included. Using January 2021 as the time cutoff, those admitted after January 2021 were included in the hold-out set (n = 20). The remaining patients were separated randomly (8:2 ratio) into training (n = 55) and validation (n = 11) sets. All preoperative and postoperative MR images, including T1-weighted (T1w), T2-weighted (T2w), fluid-attenuated inversion recovery (FLAIR), and combined (T1w + T2w + FLAIR) images, were included. The least absolute shrinkage and selection operator (LASSO) was used to select features. Multivariable logistic regression analysis was used to develop the diagnosis model. The performance of the radiomic nomogram was evaluated with an area under the curve (AUC), net reclassification improvement (NRI), integrated discrimination improvement (IDI), calibration and clinical utility.ResultsThe model-based radiomics features that were selected from combined sequences (T1w + T2w + FLAIR) had the highest performances in all models and showed better diagnostic performance than inexperienced radiologists in the training (AUCs: 0.847 VS. 0.664, p = 0.008), validation (AUC: 0.857 VS. 0.521, p = 0.155), and hold-out sets (AUCs: 0.828 VS. 0.571, p = 0.080). The positive values of NRI (0.402, 0.607, 0.424) and IDI (0.158, 0.264, 0.264) in the three sets indicated that the diagnostic performance of Model-Combined improved significantly. The radiomics nomogram fit well in calibration curves (p > 0.05), and decision curve analysis further confirmed the clinical usefulness of the nomogram. Additionally, the contrast (the radiomics feature) of the FCD lesions not only played a crucial role in the classifier but also had a significant correlation (r = -0.319, p < 0.05) with the duration of FCD.ConclusionThe radiomics nomogram generated by logistic regression model-based multiparametric MRI represents an important advancement in FCD diagnosis and treatment.

Predicting Long COVID in the National COVID Cohort Collaborative Using Super Learner: Cohort Study.

Postacute sequelae of COVID-19 (PASC), also known as long COVID, is a broad grouping of a range of long-term symptoms following acute COVID-19. These symptoms can occur across a range of biological systems, leading to challenges in determining risk factors for PASC and the causal etiology of this disorder. An understanding of characteristics that are predictive of future PASC is valuable, as this can inform the identification of high-risk individuals and future preventative efforts. However, current knowledge regarding PASC risk factors is limited. Using a sample of 55,257 patients (at a ratio of 1 patient with PASC to 4 matched controls) from the National COVID Cohort Collaborative, as part of the National Institutes of Health Long COVID Computational Challenge, we sought to predict individual risk of PASC diagnosis from a curated set of clinically informed covariates. The National COVID Cohort Collaborative includes electronic health records for more than 22 million patients from 84 sites across the United States. We predicted individual PASC status, given covariate information, using Super Learner (an ensemble machine learning algorithm also known as stacking) to learn the optimal combination of gradient boosting and random forest algorithms to maximize the area under the receiver operator curve. We evaluated variable importance (Shapley values) based on 3 levels: individual features, temporal windows, and clinical domains. We externally validated these findings using a holdout set of randomly selected study sites. We were able to predict individual PASC diagnoses accurately (area under the curve 0.874). The individual features of the length of observation period, number of health care interactions during acute COVID-19, and viral lower respiratory infection were the most predictive of subsequent PASC diagnosis. Temporally, we found that baseline characteristics were the most predictive of future PASC diagnosis, compared with characteristics immediately before, during, or after acute COVID-19. We found that the clinical domains of health care use, demographics or anthropometry, and respiratory factors were the most predictive of PASC diagnosis. The methods outlined here provide an open-source, applied example of using Super Learner to predict PASC status using electronic health record data, which can be replicated across a variety of settings. Across individual predictors and clinical domains, we consistently found that factors related to health care use were the strongest predictors of PASC diagnosis. This indicates that any observational studies using PASC diagnosis as a primary outcome must rigorously account for heterogeneous health care use. Our temporal findings support the hypothesis that clinicians may be able to accurately assess the risk of PASC in patients before acute COVID-19 diagnosis, which could improve early interventions and preventive care. Our findings also highlight the importance of respiratory characteristics in PASC risk assessment. RR2-10.1101/2023.07.27.23293272.

Automatic renal carcinoma biopsy guidance using forward-viewing endoscopic optical coherence tomography and deep learning

Percutaneous renal biopsy is commonly used for kidney cancer diagnosis. However, the biopsy procedure remains challenging in sampling accuracy. Here we introduce a forward-viewing optical coherence tomography probe for differentiating tumor and normal tissues, aiming at precise biopsy guidance. Totally, ten human kidney samples, nine of which had malignant renal carcinoma and one had benign oncocytoma, were used for system evaluation. Based on their distinct imaging features, carcinoma could be efficiently distinguished from normal renal tissues. Additionally, oncocytoma could be differentiated from carcinoma. We developed convolutional neural networks for tissue recognition. Compared to the conventional attenuation coefficient method, convolutional neural network models provided more accurate carcinoma predictions. These models reached a tissue recognition accuracy of 99.1% on a hold-out set of four kidney samples. Furthermore, they could efficiently distinguish oncocytoma from carcinoma. In conclusion, our convolutional neural network-aided endoscopic imaging platform could enhance carcinoma diagnosis during percutaneous renal biopsy procedures.

Age prediction from 12-lead electrocardiograms using deep learning: a comparison of four models on a contemporary, freely available dataset

Objective. The 12-lead electrocardiogram (ECG) is routine in clinical use and deep learning approaches have been shown to have the identify features not immediately apparent to human interpreters including age and sex. Several models have been published but no direct comparisons exist. Approach. We implemented three previously published models and one unpublished model to predict age and sex from a 12-lead ECG and then compared their performance on an open-access data set. Main results. All models converged and were evaluated on the holdout set. The best preforming age prediction model had a hold-out set mean absolute error of 8.06 years. The best preforming sex prediction model had a hold-out set area under the receiver operating curve of 0.92. Significance. We compared performance of four models on an open-access dataset.

Novel Domain Knowledge-Encoding Algorithm Enables Label-Efficient Deep Learning for Cardiac CT Segmentation to Guide Atrial Fibrillation Treatment in a Pilot Dataset.

Background: Segmenting computed tomography (CT) is crucial in various clinical applications, such as tailoring personalized cardiac ablation for managing cardiac arrhythmias. Automating segmentation through machine learning (ML) is hindered by the necessity for large, labeled training data, which can be challenging to obtain. This article proposes a novel approach for automated, robust labeling using domain knowledge to achieve high-performance segmentation by ML from a small training set. The approach, the domain knowledge-encoding (DOKEN) algorithm, reduces the reliance on large training datasets by encoding cardiac geometry while automatically labeling the training set. The method was validated in a hold-out dataset of CT results from an atrial fibrillation (AF) ablation study. Methods: The DOKEN algorithm parses left atrial (LA) structures, extracts "anatomical knowledge" by leveraging digital LA models (available publicly), and then applies this knowledge to achieve high ML segmentation performance with a small number of training samples. The DOKEN-labeled training set was used to train a nnU-Net deep neural network (DNN) model for segmenting cardiac CT in N = 20 patients. Subsequently, the method was tested in a hold-out set with N = 100 patients (five times larger than training set) who underwent AF ablation. Results: The DOKEN algorithm integrated with the nn-Unet model achieved high segmentation performance with few training samples, with a training to test ratio of 1:5. The Dice score of the DOKEN-enhanced model was 96.7% (IQR: 95.3% to 97.7%), with a median error in surface distance of boundaries of 1.51 mm (IQR: 0.72 to 3.12) and a mean centroid-boundary distance of 1.16 mm (95% CI: -4.57 to 6.89), similar to expert results (r = 0.99; p < 0.001). In digital hearts, the novel DOKEN approach segmented the LA structures with a mean difference for the centroid-boundary distances of -0.27 mm (95% CI: -3.87 to 3.33; r = 0.99; p < 0.0001). Conclusions: The proposed novel domain knowledge-encoding algorithm was able to perform the segmentation of six substructures of the LA, reducing the need for large training data sets. The combination of domain knowledge encoding and a machine learning approach could reduce the dependence of ML on large training datasets and could potentially be applied to AF ablation procedures and extended in the future to other imaging, 3D printing, and data science applications.

Evolutionary Strategies Enable Systematic and Reliable Uncertainty Quantification: A Proof-of-Concept Pilot Study on Resting-State Functional MRI Language Lateralization.

Reliable and trustworthy artificial intelligence (AI), particularly in high-stake medical diagnoses, necessitates effective uncertainty quantification (UQ). Existing UQ methods using model ensembles often introduce invalid variability or computational complexity, rendering them impractical and ineffective in clinical workflow. We propose a UQ approach based on deep neuroevolution (DNE), a data-efficient optimization strategy. Our goal is to replicate trends observed in expert-based UQ. We focused on language lateralization maps from resting-state functional MRI (rs-fMRI). Fifty rs-fMRI maps were divided into training/testing (30:20) sets, representing two labels: "left-dominant" and "co-dominant." DNE facilitated acquiring an ensemble of 100 models with high training and testing set accuracy. Model uncertainty was derived from distribution entropies over the 100 model predictions. Expert reviewers provided user-based uncertainties for comparison. Model (epistemic) and user-based (aleatoric) uncertainties were consistent in the independently and identically distributed (IID) testing set, mainly indicating low uncertainty. In a mostly out-of-distribution (OOD) holdout set, both model and user-based entropies correlated but displayed a bimodal distribution, with one peak representing low and another high uncertainty. We also found a statistically significant positive correlation between epistemic and aleatoric uncertainties. DNE-based UQ effectively mirrored user-based uncertainties, particularly highlighting increased uncertainty in OOD images. We conclude thatDNE-based UQ correlates with expert assessments, making it reliable for our use case and potentially for other radiology applications.

LGG-15. DEEP LEARNING ENABLES LONGITUDINAL RISK PREDICTION FOR PEDIATRIC LOW-GRADE GLIOMAS AFTER SURGERY

Abstract BACKGROUND Disease progression is challenging to predict following surgery for pediatric low-grade glioma (pLGG), and early progression indicators on MRI surveillance imaging would help guide management. Longitudinal surveillance imaging data may capture subtle temporal tumor changes and patterns that could inform recurrence risk but are difficult to synthesize clinically. We applied deep, self-supervised learning to longitudinal surveillance MRI to predict short-interval event-free-survival (EFS) risk from time-of-scan. METHODS We retrospectively collected data from 278 patients (2,174 scans) who underwent surgical resection for pLGG at Dana-Farber Cancer Institute/Boston Children’s Hospital (DFCI/BCH) from 1990-2022. A deep-learning model was pretrained with T2-FLAIR sequences to determine the chronological scan order for each patient (temporal pretraining). The model was fine-tuned to predict 1-year EFS from point-of-scan using a subset of scans from DFCI/BCH and then validated on a hold-out dataset (511 scans, 67 patients). Separately, the model was fine-tuned on an external dataset from the Children’s Brain Tumor Network (CBTN) and validated on a hold-out set (204 scans, 61 patients). Results were compared to a model trained without temporal pretraining. RESULTS Median follow-up and interval between scans were 73 and 6.3 months for DFCI/BCH and 20 and 5.8 months for CBTN. Temporal pretraining improved performance for 1-year EFS, for DFCI/BCH (AUC: 0.83 [0.71-0.90] vs. 0.78 [0.62-0.91]; F1-score: 0.80 vs. 0.67), and CBTN hold-out sets (AUC: 0.75 [0.58-0.90] vs. 0.53 [0.35-0.70], p=0.01; F1-score: 0.65 vs. 0.55). Increasing the number of longitudinal scans available to the model incrementally improved performance up to 7 scans/patient for DFCI/BCH and 10 scans/patient for CBTN. CONCLUSIONS Deep-learning with temporal pretraining improves the predictive analysis of tumor growth patterns in postoperative pLGG surveillance MRI, enabling accurate short-interval EFS prediction. This model could help inform patients and clinicians regarding surveillance regimen and initiation of adjuvant therapy.