Test Area Under The Curve Research Articles

CT radiomics combined with clinical and radiological factors predict hematoma expansion in hypertensive intracerebral hemorrhage.

This study aimed to establish a hematoma expansion (HE) prediction model for hypertensive intracerebral hemorrhage (HICH) patients by combining CT radiomics, clinical information, and conventional imaging signs. A retrospective continuous collection of HICH patients from three medical centers was divided into a training set (n = 555), a validation set (n = 239), and a test set (n = 77). Extract radiomics features from baseline CT plain scan images and combine them with clinical information and conventional imaging signs to construct radiomics models, clinical imaging sign models, and hybrid models, respectively. The models will be evaluated using the area under the curve (AUC), clinical decision curve analysis (DCA), net reclassification index (NRI), and integrated discrimination improvement (IDI). In the training, validation, and testing sets, the radiomics model predicts an AUC of HE of 0.885, 0.827, and 0.894, respectively, while the clinical imaging sign model predicts an AUC of HE of 0.759, 0.725, and 0.765, respectively. Glasgow coma scale score at admission, first CT hematoma volume, irregular hematoma shape, and radiomics score were used to construct a hybrid model, with AUCs of 0.901, 0.838, and 0.917, respectively. The DCA shows that the hybrid model had the highest net profit rate. Compared with the radiomics model and the clinical imaging sign model, the hybrid model showed an increase in NRI and IDI. The hybrid model based on CT radiomics combined with clinical and radiological factors can effectively individualize the evaluation of the risk of HE in patients with HICH. CT radiomics combined with clinical information and conventional imaging signs can identify HICH patients with a high risk of HE and provide a basis for clinical-targeted treatment. HE is an important prognostic factor in patients with HICH. The hybrid model predicted HE with training, validation, and test AUCs of 0.901, 0.838, and 0.917, respectively. This model provides a tool for a personalized clinical assessment of early HE risk.

GMILT: A Novel Transformer Network That Can Noninvasively Predict EGFR Mutation Status.

Noninvasively and accurately predicting the epidermal growth factor receptor (EGFR) mutation status is a clinically vital problem. Moreover, further identifying the most suspicious area related to the EGFR mutation status can guide the biopsy to avoid false negatives. Deep learning methods based on computed tomography (CT) images may improve the noninvasive prediction of EGFR mutation status and potentially help clinicians guide biopsies by visual methods. Inspired by the potential inherent links between EGFR mutation status and invasiveness information, we hypothesized that the predictive performance of a deep learning network can be improved through extra utilization of the invasiveness information. Here, we created a novel explainable transformer network for EGFR classification named gated multiple instance learning transformer (GMILT) by integrating multi-instance learning and discriminative weakly supervised feature learning. Pathological invasiveness information was first introduced into the multitask model as embeddings. GMILT was trained and validated on a total of 512 patients with adenocarcinoma and tested on three datasets (the internal test dataset, the external test dataset, and The Cancer Imaging Archive (TCIA) public dataset). The performance (area under the curve (AUC) =0.772 on the internal test dataset) of GMILT exceeded that of previously published methods and radiomics-based methods (i.e., random forest and support vector machine) and attained a preferable generalization ability (AUC =0.856 in the TCIA test dataset and AUC =0.756 in the external dataset). A diameter-based subgroup analysis further verified the efficiency of our model (most of the AUCs exceeded 0.772) to noninvasively predict EGFR mutation status from computed tomography (CT) images. In addition, because our method also identified the "core area" of the most suspicious area related to the EGFR mutation status, it has the potential ability to guide biopsies.

A pancreatic cancer risk prediction model (Prism) developed and validated on large-scale US clinical data

Pancreatic Duct Adenocarcinoma (PDAC) screening can enable early-stage disease detection and long-term survival. Current guidelines use inherited predisposition, with about 10% of PDAC cases eligible for screening. Using Electronic Health Record (EHR) data from a multi-institutional federated network, we developed and validated a PDAC RISk Model (Prism) for the general US population to extend early PDAC detection. Neural Network (PrismNN) and Logistic Regression (PrismLR) were developed using EHR data from 55 US Health Care Organisations (HCOs) to predict PDAC risk 6-18 months before diagnosis for patients 40 years or older. Model performance was assessed using Area Under the Curve (AUC) and calibration plots. Models were internal-externally validated by geographic location, race, and time. Simulated model deployment evaluated Standardised Incidence Ratio (SIR) and other metrics. With 35,387 PDAC cases, 1,500,081 controls, and 87 features per patient, PrismNN obtained a test AUC of 0.826 (95% CI: 0.824-0.828) (PrismLR: 0.800 (95% CI: 0.798-0.802)). PrismNN's average internal-external validation AUCs were 0.740 for locations, 0.828 for races, and 0.789 (95% CI: 0.762-0.816) for time. At SIR=5.10 (exceeding the current screening inclusion threshold) in simulated model deployment, PrismNN sensitivity was 35.9% (specificity 95.3%). Prism models demonstrated good accuracy and generalizability across diverse populations. PrismNN could find 3.5 times more cases at comparable risk than current screening guidelines. The small number of features provided a basis for model interpretation. Integration with the federated network provided data from a large, heterogeneous patient population and a pathway to future clinical deployment. Prevent Cancer Foundation, TriNetX, Boeing, DARPA, NSF, and Aarno Labs.

Column generation-based prototype learning for optimizing area under the receiver operating characteristic curve

The traditional classification algorithms focus on the maximization of classification accuracy which might lead to poor performance in practice by forcing classifiers to overfit to the majority class. In order to overcome this issue, various approaches focus on the optimization of alternative loss functions such as the Area Under the Curve (AUC). AUC is a Receiver Operating Characteristics (ROC) metric that has been widely used to measure classification performance, especially when there are class imbalances. In this work, we propose a column generation (CG)-based algorithm called Ranking-CG, which learns a model, similar to the popular Ranking SVM, through approximate maximization of the AUC. Unlike the Ranking SVM, our algorithm utilizes a column generation method that iteratively adds features to control the model complexity effectively working as an internal feature selection procedure. Our experiments show that column generation can be an important tool to prevent overfitting. We extend the Ranking-CG by proposing a prototype generation method, denoted by Ranking-CG Prototype, that constructs reference points by solving a non-linear optimization problem. Based on the extensive experiments conducted on 74 binary classification problems, the Ranking-CG Prototype yields the best average test AUC among all competing methods by using significantly few features than other benchmarks.

Identification of an Ultra-High-Risk Subgroup of Neuroblastoma Patients within the High-Risk Cohort Using a Computed Tomography-Based Radiomics Approach

To identify ultra-high-risk (UHR) neuroblastoma patients who experienced disease-related mortality within 18 months of diagnosis within the high-risk cohort using computed tomography (CT)-based radiomics analysis. A retrospective analysis was conducted on 105 high-risk neuroblastoma patients, divided into a training set (n=74) and a test set (n=31). Radiomics features were extracted and selected from arterial phase CT images, and an optimal radiomics signature was established using the support vector machine algorithm. Evaluation metrics, including area under the curve (AUC) and 95% confidence interval (CI), were calculated. Furthermore, the fit and clinical benefit of the signature, along with its correlation with overall survival (OS), were analyzed. The optimal radiomics signature comprised 11 features. In the training set, AUC and accuracy were 0.911 (95% CI: 0.840-0.982) and 0.892, respectively. In the test set, AUC and accuracy were 0.828 (95% CI: 0.669-0.987) and 0.839, respectively. There was no significant difference between predicted probability and actual probability, and the signature demonstrated net benefit. The concordance index of this signature for predicting OS was 0.743 (95% CI: 0.672-0.814) in the training set and 0.688 (95% CI: 0.566-0.810) in the test set. Moreover, the signature achieved AUC values of 0.832, 0.863, and 0.721 for 1-year, 2-year, and 3-year OS in the training set, and 0.870, 0.836, and 0.638 in the test set for the respective time periods. The utilization of CT-based radiomics signature to identify an UHR subgroup of neuroblastoma patients within the high-risk cohort can help aid in predicting early disease progression.

Association of red blood cell distribution width with hospital admission and in-hospital mortality across all-cause adult emergency department visits.

To test the association between the initial red blood cell distribution width (RDW) value in the emergency department (ED) and hospital admission and, among those admitted, in-hospital mortality. We perform a retrospective analysis of 210 930 adult ED visits with complete blood count results from March 2013 to February 2022. Primary outcomes were hospital admission and in-hospital mortality. Variables for each visit included demographics, comorbidities, vital signs, basic metabolic panel, complete blood count, and final diagnosis. The association of each outcome with the initial RDW value was calculated across 3 age groups (<45, 45-65, and >65) as well as across 374 diagnosis categories. Logistic regression (LR) and XGBoost models using all variables excluding final diagnoses were built to test whether RDW was a highly weighted and informative predictor for each outcome. Finally, simplified models using only age, sex, and vital signs were built to test whether RDW had additive predictive value. Compared to that of discharged visits (mean [SD]: 13.8 [2.03]), RDW was significantly elevated in visits that resulted in admission (15.1 [2.72]) and, among admissions, those resulting in intensive care unit stay (15.3 [2.88]) and/or death (16.8 [3.25]). This relationship held across age groups as well as across various diagnosis categories. An RDW >16 achieved 90% specificity for hospital admission, while an RDW >18.5 achieved 90% specificity for in-hospital mortality. LR achieved a test area under the curve (AUC) of 0.77 (95% confidence interval [CI] 0.77-0.78) for hospital admission and 0.85 (95% CI 0.81-0.88) for in-hospital mortality, while XGBoost achieved a test AUC of 0.90 (95% CI 0.89-0.90) for hospital admission and 0.96 (95% CI 0.94-0.97) for in-hospital mortality. RDW had high scaled weights and information gain for both outcomes and had additive value in simplified models predicting hospital admission. Elevated RDW, previously associated with mortality in myocardial infarction, pulmonary embolism, heart failure, sepsis, and COVID-19, is associated with hospital admission and in-hospital mortality across all-cause adult ED visits. Used alone, elevated RDW may be a specific, but not sensitive, test for both outcomes, with multivariate LR and XGBoost models showing significantly improved test characteristics. RDW, a component of the complete blood count panel routinely ordered as the initial workup for the undifferentiated patient, may be a generalizable biomarker for acuity in the ED.

Prediction of prostate tumour hypoxia using pre-treatment MRI-derived radiomics: preliminary findings

PurposeTo develop a machine learning (ML) model based on radiomic features (RF) extracted from whole prostate gland magnetic resonance imaging (MRI) for prediction of tumour hypoxia pre-radiotherapy.Material and methodsConsecutive patients with high-grade prostate cancer and pre-treatment MRI treated with radiotherapy between 01/12/2007 and 1/08/2013 at two cancer centres were included. Cancers were dichotomised as normoxic or hypoxic using a biopsy-based 32-gene hypoxia signature (Ragnum signature). Prostate segmentation was performed on axial T2-weighted (T2w) sequences using RayStation (v9.1). Histogram standardisation was applied prior to RF extraction. PyRadiomics (v3.0.1) was used to extract RFs for analysis. The cohort was split 80:20 into training and test sets. Six different ML classifiers for distinguishing hypoxia were trained and tuned using five different feature selection models and fivefold cross-validation with 20 repeats. The model with the highest mean validation area under the curve (AUC) receiver operating characteristic (ROC) curve was tested on the unseen set, and AUCs were compared via DeLong test with 95% confidence interval (CI).Results195 patients were included with 97 (49.7%) having hypoxic tumours. The hypoxia prediction model with best performance was derived using ridge regression and had a test AUC of 0.69 (95% CI: 0.14). The test AUC for the clinical-only model was lower (0.57), but this was not statistically significant (p = 0.35). The five selected RFs included textural and wavelet-transformed features.ConclusionWhole prostate MRI-radiomics has the potential to non-invasively predict tumour hypoxia prior to radiotherapy which may be helpful for individualised treatment optimisation.

A Deep Learning Pipeline Using Prior Knowledge for Automatic Evaluation of Placenta Accreta Spectrum Disorders With MRI.

The diagnosis of prenatal placenta accreta spectrum (PAS) with magnetic resonance imaging (MRI) is highly dependent on radiologists' experience. A deep learning (DL) method using the prior knowledge that PAS-related signs are generally found along the utero-placental borderline (UPB) may help radiologists, especially those with less experience, to mitigate this issue. To develop a DL tool for antenatal diagnosis of PAS using T2-weighted MR images. Retrospective. Five hundred and forty pregnant women with clinically suspected PAS disorders from two institutions, divided into training (409), internal test (103), and external test (28) datasets. Sagittal T2-weighted fast spin echo sequence at 1.5 T and 3 T. An nnU-Net was trained for placenta segmentation. The UPB straightening approach was used to extract the utero-placental boundary region. The UPB image was then fed into DenseNet-PAS for PAS diagnosis. DenseNet-PP learnt placental position information to improve the PAS diagnosis performance. Three radiologists with 8, 10, and 12 years of experience independently evaluated the images. Two radiologists marked the placenta tissue. Histopathological findings were the reference standard. Area under the curve (AUC) was used to evaluate the classification. Dice coefficient evaluated the segmentation between radiologists and the model performance. The Mann-Whitney U-test or the chi-squared test assessed the significance of differences. Decision curve analysis was used to determine clinical effectiveness. DeLong's test was used to compare AUCs. Of the 540 patients, 170 had PAS disorders confirmed by histopathology. The DL model using UPB images and placental position yielded the highest AUC of 0.860 and 0.897 in internal test and external test cohorts, respectively, significantly exceeding the performance of three radiologists (internal test AUC, 0.737-0.770). By extracting the UPB image, this fully automatic DL pipeline achieved high accuracy and may assist radiologists in PAS diagnosis using MRI. 3 TECHNICAL EFFICACY: Stage 2.

Evaluation of Machine Learning to Detect Influenza Using Wearable Sensor Data and Patient-Reported Symptoms: Cohort Study.

Machine learning offers quantitative pattern recognition analysis of wearable device data and has the potential to detect illness onset and monitor influenza-like illness (ILI) in patients who are infected. This study aims to evaluate the ability of machine-learning algorithms to distinguish between participants who are influenza positive and influenza negative in a cohort of symptomatic patients with ILI using wearable sensor (activity) data and self-reported symptom data during the latent and early symptomatic periods of ILI. This prospective observational cohort study used the extreme gradient boosting (XGBoost) classifier to determine whether a participant was influenza positive or negative based on 3 models using symptom-only data, activity-only data, and combined symptom and activity data. Data were collected from the Home Testing of Respiratory Illness (HTRI) study and FluStudy2020, both conducted between December 2019 and October 2020. The model was developed using the FluStudy2020 data and tested on the HTRI data. Analyses included participants in these studies with an at-home influenza diagnostic test result. Fitbit (Google LLC) devices were used to measure participants' steps, heart rate, and sleep parameters. Participants detailed their ILI symptoms, health care-seeking behaviors, and quality of life. Model performance was assessed by area under the curve (AUC), balanced accuracy, recall (sensitivity), specificity, precision (positive predictive value), negative predictive value, and weighted harmonic mean of precision and recall (F2) score. An influenza diagnostic test result was available for 953 and 925 participants in HTRI and FluStudy2020, respectively, of whom 848 (89%) and 840 (90.8%) had activity data. For the training and validation sets, the highest performing model was trained on the combined symptom and activity data (training AUC=0.77; validation AUC=0.74) versus symptom-only (training AUC=0.73; validation AUC=0.72) and activity-only (training AUC=0.68; validation AUC=0.65) data. For the FluStudy2020 test set, the performance of the model trained on combined symptom and activity data was closely aligned with that of the symptom-only model (combined symptom and activity test AUC=0.74; symptom-only test AUC=0.74). These results were validated using independent HTRI data (combined symptom and activity evaluation AUC=0.75; symptom-only evaluation AUC=0.74). The top features guiding influenza detection were cough; mean resting heart rate during main sleep; fever; total minutes in bed for the combined model; and fever, cough, and sore throat for the symptom-only model. Machine-learning algorithms had moderate accuracy in detecting influenza, suggesting that previous findings from research-grade sensors tested in highly controlled experimental settings may not easily translate to scalable commercial-grade sensors. In the future, more advanced wearable sensors may improve their performance in the early detection and discrimination of viral respiratory infections.

CVD22: Explainable artificial intelligence determination of the relationship of troponin to D-Dimer, mortality, and CK-MB in COVID-19 patients

Background and purposeCOVID-19, which emerged in Wuhan (China), is one of the deadliest and fastest-spreading pandemics as of the end of 2019. According to the World Health Organization (WHO), there are more than 100 million infectious cases worldwide. Therefore, research models are crucial for managing the pandemic scenario. However, because the behavior of this epidemic is so complex and difficult to understand, an effective model must not only produce accurate predictive results but must also have a clear explanation that enables human experts to act proactively. For this reason, an innovative study has been planned to diagnose Troponin levels in the COVID-19 process with explainable white box algorithms to reach a clear explanation. MethodsUsing the pandemic data provided by Erzurum Training and Research Hospital (decision number: 2022/13-145), an interpretable explanation of Troponin data was provided in the COVID-19 process with SHApley Additive exPlanations (SHAP) algorithms. Five machine learning (ML) algorithms were developed. Model performances were determined based on training, test accuracies, precision, F1-score, recall, and AUC (Area Under the Curve) values. Feature importance was estimated according to Shapley values by applying the SHApley Additive exPlanations (SHAP) method to the model with high accuracy. The model created with Streamlit v.3.9 was integrated into the interface with the name CVD22. ResultsAmong the five-machine learning (ML) models created with pandemic data, the best model was selected with the values of 1.0, 0.83, 0.86, 0.83, 0.80, and 0.91 in train and test accuracy, precision, F1-score, recall, and AUC values, respectively. As a result of feature selection and SHApley Additive exPlanations (SHAP) algorithms applied to the XGBoost model, it was determined that DDimer mean, mortality, CKMB (creatine kinase myocardial band), and Glucose were the features with the highest importance over the model estimation. ConclusionsRecent advances in new explainable artificial intelligence (XAI) models have successfully made it possible to predict the future using large historical datasets. Therefore, throughout the ongoing pandemic, CVD22 (https://cvd22covid.streamlitapp.com/) can be used as a guide to help authorities or medical professionals make the best decisions quickly.

Yiqi Huazhuo decoction increases insulin secretion in type 2 diabetic rats by regulating the pancreatic GPR40-IP3R-1 signaling pathway.

Objective: Yiqi Huazhuo Decoction (YD) reduces blood glucose, glycated hemoglobin, body weight, and insulin resistance in patients with type 2 diabetes mellitus (T2DM), but its exact mechanisms are unknown. This study investigated the therapeutic effects and mechanisms of YD on impaired insulin secretion in T2DM rats. Methods: T2DM rats were randomized to the model, YD-lo (15mg/kg/d YD, 10weeks), YD-hi (30mg/kg/d YD, 10 weeks), positive drug (TAK-875), and healthy control groups. The rats underwent an oral glucose tolerance test (OGTT), glucose-stimulated insulin secretion (GSIS) test, and serum lipid measurements. High-fat and high-glucose-injured RIN-m5f cells were treated with YD (30 or 150mg/mL) for 48h. GPR40 and IP3R-1 expression levels were determined by immunofluorescence, qRT-PCR, and western blot. Results: Compared with the model group, the OGTT area under the curve (AUC) in the YD-hi group was decreased by 26.7%, the insulin release test (IRT) AUC in the YD-hi group was increased by 45.9%, and the GSIS AUC was increased by 33.9% (p < 0.05). Compared with the model cells, the insulin secretion after glucose stimulation in the YD-hi group was increased by 24.5%, similar to the TAK-875 group (23.1%) (p > 0.05). GPR40 and IP3R-1 mRNA in the model cells were decreased by 49.5% and 51.2% compared with the control cells (p < 0.05). In the YD-hi group, GPR40 and IP3R-1 mRNA levels were increased by 58.1% and 39.3% (p < 0.05), similar to the TAK-875 group. The changes in protein expression were similar to mRNA. Conclusion: YD promotes insulin secretion from pancreatic islet β-cell in T2DM rats by regulating the GPR40-IP3R-1 pathway, thereby reducing blood glucose.

Machine-Learning Model for the Prediction of Hypoxaemia during Endoscopic Retrograde Cholangiopancreatography under Monitored Anaesthesia Care.

Hypoxaemia is a significant adverse event during endoscopic retrograde cholangiopancreatography (ERCP) under monitored anaesthesia care (MAC); however, no model has been developed to predict hypoxaemia. We aimed to develop and compare logistic regression (LR) and machine learning (ML) models to predict hypoxaemia during ERCP under MAC. We collected patient data from our institutional ERCP database. The study population was randomly divided into training and test sets (7:3). Models were fit to training data and evaluated on unseen test data. The training set was further split into k-fold (k=5) for tuning hyperparameters, such as feature selection and early stopping. Models were trained over k loops; the i-th fold was set aside as a validation set in the i-th loop. Model performance was measured using area under the curve (AUC). We identified 6114 cases of ERCP under MAC, with a total hypoxaemia rate of 5.9%. The LR model was established by combining eight variables and had a test AUC of 0.693. The ML and LR models were evaluated on 30 independent data splits. The average test AUC for LR was 0.7230, which improved to 0.7336 by adding eight more variables with an l1 regularisation-based selection technique and ensembling the LRs and gradient boosting algorithm (GBM). The high-risk group was discriminated using the GBM ensemble model, with a sensitivity and specificity of 63.6% and 72.2%, respectively. We established GBM ensemble model and LR model for risk prediction, which demonstrated good potential for preventing hypoxaemia during ERCP under MAC.

Predicting Tumor Perineural Invasion Status in High-Grade Prostate Cancer Based on a Clinical–Radiomics Model Incorporating T2-Weighted and Diffusion-Weighted Magnetic Resonance Images

To explore the role of bi-parametric MRI radiomics features in identifying PNI in high-grade PCa and to further develop a combined nomogram with clinical information. 183 high-grade PCa patients were included in this retrospective study. Tumor regions of interest (ROIs) were manually delineated on T2WI and DWI images. Radiomics features were extracted from lesion area segmented images obtained. Univariate logistic regression analysis and the least absolute shrinkage and selection operator (LASSO) method were used for feature selection. A clinical model, a radiomics model, and a combined model were developed to predict PNI positive. Predictive performance was estimated using receiver operating characteristic (ROC) curves, calibration curves, and decision curves. The differential diagnostic efficiency of the clinical model had no statistical difference compared with the radiomics model (area under the curve (AUC) values were 0.766 and 0.823 in the train and test group, respectively). The radiomics model showed better discrimination in both the train cohort and test cohort (train AUC: 0.879 and test AUC: 0.908) than each subcategory image (T2WI train AUC: 0.813 and test AUC: 0.827; DWI train AUC: 0.749 and test AUC: 0.734). The discrimination efficiency improved when combining the radiomics and clinical models (train AUC: 0.906 and test AUC: 0.947). The model including radiomics signatures and clinical factors can accurately predict PNI positive in high-grade PCa patients.

568. Using Machine Learning to Predict Place-Based Risks for Staphylococcus aureus Infections in Children

Abstract Background Community-onset Staphylococcus aureus (CO-S. aureus) pediatric infections, methicillin-resistant S. aureus (MRSA) and methicillin-susceptible S. aureus (MSSA) continue to contribute to the burden of infections seen in the ambulatory setting in the US. Individual risk factors have been identified, but place-based factors and specific geographic locality have not been well-studied. The purpose of this study is to predict place-based factors that contribute to the spread of CO-S. aureus in a major urban area using maximum entropy (MaxEnt), a machine learning technique. Methods Electronic medical records from two pediatric hospitals (2002 to 2016) were retrospectively reviewed. Inclusion criteria: a confirmed S. aureus infection within 48 hours of hospital admission (CO-S. aureus), &amp;lt; 19 years old, and a geo-referenced address within Atlanta’s metropolitan statistical area (MSA). Fourteen place-based factors, at the US Census block group level, were included in the MaxEnt models: &amp;lt; 18 years old, Caucasian, African American, ethnicity, poverty, education attainment, crowding, daycare, kindergarten enrollment, distance to K-12 school, distance to a children’s hospital, distance to a daycare center, and population density. A total of four models (CO-MRSA early, CO-MSSA early, CO-MRSA later, and CO-MSSA later) were run using the MaxEnt software. For each model, 75% and 25% of data was randomly assigned to training and testing groups, respectively. Models were assessed by jack-knife tests. Results 16,124 records met eligibility criteria for MaxEnt models. The training Area Under the Curve (AUC) ranged from 0.771 to 0.837 and the test AUC ranged from 0.769 to 0.804. Population density had the highest contribution in predicting CO-MRSA and CO-MSSA locations, which was confirmed by jack-knife tests. Conclusion By applying MaxEnt to pediatric CO-S. aureus infections in the Atlanta MSA, it was found that higher risks of CO-S. aureus infections may exist in more densely populated areas. MaxEnt can be utilized to identify potential future areas of CO-MRSA and CO-MSSA infections based on estimated or predicted changes to the place-based factors used to build these models. Disclosures Lilly Immergluck, MD, MS, GSK: Clinical Trial- PI|Merck: Vaccine Trial Site- serve as PI|Moderna: Board Member|Novavax: Part of CoVID-19 Phase 3 Trial through US Covid Prevention Network.

CT-based deep learning radiomics signature for the preoperative prediction of the muscle-invasive status of bladder cancer.

Although the preoperative assessment of whether a bladder cancer (BCa) indicates muscular invasion is crucial for adequate treatment, there currently exist some challenges involved in preoperative diagnosis of BCa with muscular invasion. The aim of this study was to construct deep learning radiomic signature (DLRS) for preoperative predicting the muscle invasion status of BCa. A retrospective review covering 173 patients revealed 43 with pathologically proven muscle-invasive bladder cancer (MIBC) and 130 with non-muscle-invasive bladder cancer (non- MIBC). A total of 129 patients were randomly assigned to the training cohort and 44 to the test cohort. The Pearson correlation coefficient combined with the least absolute shrinkage and selection operator (LASSO) was utilized to reduce radiomic redundancy. To decrease the dimension of deep learning features, Principal Component Analysis (PCA) was adopted. Six machine learning classifiers were finally constructed based on deep learning radiomics features, which were adopted to predict the muscle invasion status of bladder cancer. The area under the curve (AUC), accuracy, sensitivity and specificity were used to evaluate the performance of the model. According to the comparison, DLRS-based models performed the best in predicting muscle violation status, with MLP (Train AUC: 0.973260 (95% CI 0.9488-0.9978) and Test AUC: 0.884298 (95% CI 0.7831-0.9855)) outperforming the other models. In the test cohort, the sensitivity, specificity and accuracy of the MLP model were 0.91 (95% CI 0.551-0.873), 0.78 (95% CI 0.594-0.863) and 0.58 (95% CI 0.729-0.827), respectively. DCA indicated that the MLP model showed better clinical utility than Radiomics-only model, which was demonstrated by the decision curve analysis. A deep radiomics model constructed with CT images can accurately predict the muscle invasion status of bladder cancer.

Deploying deep learning models on unseen medical imaging using adversarial domain adaptation.

The fundamental challenge in machine learning is ensuring that trained models generalize well to unseen data. We developed a general technique for ameliorating the effect of dataset shift using generative adversarial networks (GANs) on a dataset of 149,298 handwritten digits and dataset of 868,549 chest radiographs obtained from four academic medical centers. Efficacy was assessed by comparing area under the curve (AUC) pre- and post-adaptation. On the digit recognition task, the baseline CNN achieved an average internal test AUC of 99.87% (95% CI, 99.87-99.87%), which decreased to an average external test AUC of 91.85% (95% CI, 91.82-91.88%), with an average salvage of 35% from baseline upon adaptation. On the lung pathology classification task, the baseline CNN achieved an average internal test AUC of 78.07% (95% CI, 77.97-78.17%) and an average external test AUC of 71.43% (95% CI, 71.32-71.60%), with a salvage of 25% from baseline upon adaptation. Adversarial domain adaptation leads to improved model performance on radiographic data derived from multiple out-of-sample healthcare populations. This work can be applied to other medical imaging domains to help shape the deployment toolkit of machine learning in medicine.

Utility of pre-treatment FDG PET/CT–derived machine learning models for outcome prediction in classical Hodgkin lymphoma

ObjectivesRelapse occurs in ~20% of patients with classical Hodgkin lymphoma (cHL) despite treatment adaption based on 2-deoxy-2-[18F]fluoro-d-glucose positron emission tomography/computed tomography response. The objective was to evaluate pre-treatment FDG PET/CT–derived machine learning (ML) models for predicting outcome in patients with cHL.MethodsAll cHL patients undergoing pre-treatment PET/CT at our institution between 2008 and 2018 were retrospectively identified. A 1.5 × mean liver standardised uptake value (SUV) and a fixed 4.0 SUV threshold were used to segment PET/CT data. Feature extraction was performed using PyRadiomics with ComBat harmonisation. Training (80%) and test (20%) cohorts stratified around 2-year event-free survival (EFS), age, sex, ethnicity and disease stage were defined. Seven ML models were trained and hyperparameters tuned using stratified 5-fold cross-validation. Area under the curve (AUC) from receiver operator characteristic analysis was used to assess performance.ResultsA total of 289 patients (153 males), median age 36 (range 16–88 years), were included. There was no significant difference between training (n = 231) and test cohorts (n = 58) (p value > 0.05). A ridge regression model using a 1.5 × mean liver SUV segmentation had the highest performance, with mean training, validation and test AUCs of 0.82 ± 0.002, 0.79 ± 0.01 and 0.81 ± 0.12. However, there was no significant difference between a logistic model derived from metabolic tumour volume and clinical features or the highest performing radiomic model.ConclusionsOutcome prediction using pre-treatment FDG PET/CT–derived ML models is feasible in cHL patients. Further work is needed to determine optimum predictive thresholds for clinical use.Key points• A fixed threshold segmentation method led to more robust radiomic features.• A radiomic-based model for predicting 2-year event-free survival in classical Hodgkin lymphoma patients is feasible.• A predictive model based on ridge regression was the best performing model on our dataset.

#70 Predicting place-based factors for Staphylococcus aureus infections within pediatric population using machine learning

Abstract Background Community-onset Staphylococcus aureus (CO-S. aureus) infections, methicillin-resistant S. aureus (MRSA) and methicillin-susceptible S. aureus (MSSA), have a high burden on United States (US) Emergency Departments and urgent care, particularly within the pediatric population. Although individual risk factors have been well-studied, the influence of specific geographic locality and place-based risks for CO-S. aureus infections have not been well-characterized. Maximum entropy (MaxEnt) is a machine learning technique for ecological niche modeling, which predicts the distribution of disease vectors and their possible disease transmissions using environmental and other relevant risk factors. The aim of this study is to predict socioecological factors that contribute to the spread of CO-S. aureus in a major urban area. Method Electronic medical records from children with staphylococcal infections, who were treated at two pediatric hospitals from 2002 to 2016, were retrospectively reviewed. Children were included in the analyses if they had a confirmed S. aureus infection within 48 hours of hospital admission (i.e., CO-S. aureus), less than 19 years old, and a geo-referenced address within Atlanta’s metropolitan statistical area (MSA). The timespan was divided into two periods, 2002-2005 (early) and 2006-2010 (later), mimicking the trend of CO-S. aureus. Fourteen place-based factors, obtained at the US Census Bureau block group level, were included in the MaxEnt model: population under 18 years old, Caucasian, African American, ethnicity, poverty, low education (high school diploma), high education (Bachelor’s degree and above), crowding, nursery school enrollment, kindergarten enrollment, distance to K-12 school, distance to a children’s hospital, distance to a daycare center, and population density. These were based on factors previously determined a priori. A total of four models (CO-MRSA early, CO-MSSA early, CO-MRSA later, and CO-MSSA later) were run using the MaxEnt software (v.3.4.1). For each model, 75% and 25% of data was randomly assigned to training and testing groups, respectively. Models were assessed by jack-knife tests. Results 16,124 records met the eligibility criteria and were included in the MaxEnt models. Preliminary analyses of data from 2002-2010 suggest training Area Under the Curve (AUC) ranging from 0.802 to 0.828 and the test AUC ranging from 0.796 to 0.809, demonstrating these models are performing very well. Population density had the highest contribution in predicting CO-MRSA and CO-MSSA locations, which was confirmed by jack-knife tests. Conclusion By applying MaxEnt to pediatric CO-S. aureus infections in the Atlanta MSA, it was found that higher risks of CO-S. aureus infections may exist in more densely populated areas. MaxEnt can be utilized to identify potential future areas of CO-MRSA and CO-MSSA infections based on estimated or predicted changes to the place-based factors used to build these models, most notably population density. Predicted risk areas should have more frequent monitoring to prevent S. aureus infection outbreaks, which will also allow for more time to pool public health resources for these areas to quickly and effectively control outbreaks.

Discovery of Pre-Treatment FDG PET/CT-Derived Radiomics-Based Models for Predicting Outcome in Diffuse Large B-Cell Lymphoma.

Simple SummaryDiffuse large B-cell lymphoma (DLBCL) is the most common type of lymphoma. Even with the improvements in the treatment of DLBCL, around a quarter of patients will experience recurrence. The aim of this single centre retrospective study was to predict which patients would have recurrence within 2 years of their treatment using machine learning techniques based on radiomics extracted from the staging PET/CT images. Our study demonstrated that in our dataset of 229 patients (training data = 183, test data = 46) that a combined radiomic and clinical based model performed better than a simple model based on metabolic tumour volume, and that it had a good predictive ability which was maintained when tested on an unseen test set.Background: Approximately 30% of patients with diffuse large B-cell lymphoma (DLBCL) will have recurrence. The aim of this study was to develop a radiomic based model derived from baseline PET/CT to predict 2-year event free survival (2-EFS). Methods: Patients with DLBCL treated with R-CHOP chemotherapy undergoing pre-treatment PET/CT between January 2008 and January 2018 were included. The dataset was split into training and internal unseen test sets (ratio 80:20). A logistic regression model using metabolic tumour volume (MTV) and six different machine learning classifiers created from clinical and radiomic features derived from the baseline PET/CT were trained and tuned using four-fold cross validation. The model with the highest mean validation receiver operator characteristic (ROC) curve area under the curve (AUC) was tested on the unseen test set. Results: 229 DLBCL patients met the inclusion criteria with 62 (27%) having 2-EFS events. The training cohort had 183 patients with 46 patients in the unseen test cohort. The model with the highest mean validation AUC combined clinical and radiomic features in a ridge regression model with a mean validation AUC of 0.75 ± 0.06 and a test AUC of 0.73. Conclusions: Radiomics based models demonstrate promise in predicting outcomes in DLBCL patients.

Feasibility, Tolerability, and Patient-Reported Outcomes with Pharmacokinetic (PK)-Directed Dosing of Evomela® (propylene glycol free melphalan) for Multiple Myeloma and AL Amyloidosis Patients Undergoing Autologous Hematopoietic Stem Cell Transplant (AHCT)

Background: For multiple myeloma (MM) and AL amyloidosis (AL) patients undergoing autologous hematopoietic stem cell transplant (AHCT), high-dose melphalan is given at a fixed dose based on body surface area with five-fold inter-patient variability in exposure, as measured by area under the curve (AUC). Patients with higher than the median AUC (12.85mg/L*h) have prolonged survival, but increased toxicity (Shaw et al., BBMT 2012). Pharmacokinetically (PK)-directed dosing has the potential to achieve an optimal concentration in each patient. We prospectively evaluated the feasibility of PK-directed dosing of propylene glycol free melphalan (Evomela®, PGF-MEL), a more stable and possibly less toxic intravenous formulation of melphalan, to personalize therapy for maximum efficacy and tolerability.Methods: MM and AL patients age 18-75 received a 10mg/m2 test dose of PGF-MEL within 14 days of AHCT. Serum samples were drawn at 5, 15, 30, 40, 75, and 150 min after PGF-MEL administration. Liquid chromatography tandem mass spectrometry was used to measure MEL concentrations. AUC PK modeling with Phoenix WinNonlin v6.4 was used to determine a target dose to achieve an AUC of 13.5mg/L*h (+/-1), which was given on day -2 prior to AHCT. PK samples were collected after the target dose at the same time points as after the test dose. AHCT with 3-10 x10^6 CD34+ cells/kg was conducted per institutional guidelines. Patient-reported symptom burden was collected prospectively using the MD Anderson Symptom Inventory - MM (MDASI-MM) on day +1, 6, 11, 30, 60, and 90. Toxicities were collected by chart review through day +100 after AHCT and compared using either a T-test or Fisher's exact test between patients above and below AUC 14.5 mg/L*h.Results: Ten patients (7 MM, 3 AL) with 50% male, median age 67yr (range 53-74), and median body surface area 1.95 (range 1.56-2.35) had a median of 4.5 days (range 1-24) between the test and target doses of PGF-MEL. Target doses were calculated for 90% of patients and 70% received the calculated dose. The reasons for not administering the calculated dose were inability to calculate a test dose AUC (N=1), toxicity after the test dose (N=1), and target dose outside study safety parameters (N=1). Median test and target dose AUCs were 0.655 (range 0.447-0.869) and 15.65mg/L*h (range 10.1-17.2, Figure 1), respectively. No patients receiving the calculated target dose had an AUC within the goal range. MDASI-MM AUCs were calculated for fatigue, pain, sleep, appetite, drowsiness, and the average of these 5 symptoms. Overall, higher scores were seen in patients with MEL AUCs above 14.5mg/L*h compared with below (Figure 2). More patients with higher AUCs had fevers (83 vs 50%), ≥500ml of diarrhea per day (83 vs 75%), grade ≥3 mucositis (50 vs 25%), and need for patient-controlled analgesia (33 vs 25%), but these were not statistically significantly different between the groups (Table 1). Neutrophil engraftment occurred at a median of 10 (range 9-11) days post AHCT in both groups; patients with AUCs above 14.5mg/L*h required more pRBC and platelet transfusions. With a median follow-up of 217 days (range 179-250) in surviving patients, no patients progressed, and one AL patient died of failure to thrive at 134 days post AHCT.Conclusion: Collection of PK levels and calculation of a target dose was feasible. However, using a 10mg/m2 test dose we did not achieve the target AUC in our patients. This could be due to the small test dose which may have been insufficient to saturate the two-compartment model, rendering the PK analysis inaccurate. Patients with higher AUCs required more transfusions, had more toxicities, and had prolonged patient-reported symptom burden. Due to the small number of patients, these differences were not statistically significant, but suggest there may be an ideal concentration to minimize toxicity and maximize efficacy. Disease response and outcome data continue to be collected. Further study is needed to determine the optimum test dose and schedule, and modifications to the schema are ongoing. [Display omitted] DisclosuresKoehne:Atara: Consultancy, Patents & Royalties. Landau:Janssen: Honoraria; Takeda: Research Funding; Spectrum Pharmaceuticals: Other: Advisory Board, Research Funding; Celgene: Other: Advisory Board; Pfizer: Honoraria; Karyopharm: Consultancy; Amgen: Research Funding.