Cross-validated Area Under The Curve Research Articles

Abstract PD16-08: Title: Characterizing Changes in Tumor Heterogeneity via Radiomic Phenotyping for Predicting Response to Neoadjuvant Chemotherapy for Locally Advanced Breast Cancer: Preliminary Results from the ACRIN 6698/I-SPY 2 trial

Abstract Purpose: To predict pathologic complete response (pCR) in breast cancer patients undergoing neoadjuvant chemotherapy (NAC), from baseline and early-treatment DCE-MRI scans, in the context of the ACRIN 6698/I-SPY 2 BMMR2 challenge. Materials and Methods: The BMMR2 dataset consists of 191 patients undergoing NAC for locally advanced breast cancer as part of the ACRIN 6698/I-SPY 2 trial. DCE-MRI was obtained at time points T0 (pre-NAC), T1 (3 weeks), and T2 (12 weeks). The BMMR2 challenge provided the MRI scans, tumor annotations, and limited clinical and demographic information. The data were split 60/40; using the 60% training data, the task was to develop models to predict pCR; the competition was for best area under the curve (AUC) when applied to the 40% unseen test data. Using the publicly available CaPTk software we calculated 3 types of radiomic features within the segmented tumor volume: 1) texture of the signal enhancement ratio (SER) kinetic map of T0 images; 2) texture of the difference between the T1 kinetic maps (PE, WIS, WOS, and SER) and corresponding T0 maps; 3) texture of the difference between the T1 kinetic maps and the corresponding T0 maps, with T1 scans deformably registered to T0 scans. ComBat harmonization was applied to the extracted features to account for MRI acquisition differences. We computed the tumor longest diameter, functional tumor volume (FTV), and clinical tumor size each at T0 and T1. We modeled pCR via logistic regression. Using the training data alone, with the criteria of performance in univariable modeling and low collinearity, we selected radiomic features and clinical, demographic, and size covariates. We then performed PCA on the combined set of selected radiomic features and size covariates. We evaluated multivariable models including the selected clinical covariates in combination with the first few PCs via cross-validated AUC (5-fold, 200 repetitions) on the training data. The best models were submitted for independent evaluation on the unseen test data of the BMMR2 challenge. Results: Of the available clinical covariates, only hormone receptor (HR)± and human epidermal growth factor receptor 2 (HER2)± had any association with pCR. We retained these in all models, and performed PCA on the set combining the best-performing features and the size variables FTV at T0, FTV at T1, and longest diameter at T1. Models based on the first few PCs, HR, and HER2, had training AUCs in 0.78–0.81. Our best-performing model had an AUC on test data of 0.84, using the covariates PCs 1–5, HR, and HER2 (Table 1). Conclusions: Our preliminary results suggest that radiomic phenotyping of changes in tumor heterogeneity can accurately predict pCR early in the course of NAC. Future analysis with larger samples from ISPY-2 could also examine the effect of different therapies, including targeted therapy and immunotherapy. Table 1: Performance of candidate logistic regression models on training and test data. AUC: Area under receiver operating characteristic curve. * Mean 5-fold cross-validated AUC across 200 replicates. † Competition best-performing predictions. Citation Format: Eric A. Cohen, Rhea D. Chitalia, Snekha Thakran, Walter C. Mankowski, Alex Anh-Tu Nguyen, Hannah Horng, Elizabeth S. McDonald, Michael Feldman, Angela DeMichele, Despina Kontos. Title: Characterizing Changes in Tumor Heterogeneity via Radiomic Phenotyping for Predicting Response to Neoadjuvant Chemotherapy for Locally Advanced Breast Cancer: Preliminary Results from the ACRIN 6698/I-SPY 2 trial [abstract]. In: Proceedings of the 2022 San Antonio Breast Cancer Symposium; 2022 Dec 6-10; San Antonio, TX. Philadelphia (PA): AACR; Cancer Res 2023;83(5 Suppl):Abstract nr PD16-08.

Abstract 15372: Machine Learning Distinguishes Continuous Risk of Gastrointestinal Bleeding, Stroke, and Death in Ambulatory Lvad Patients

Introduction: Advanced data analytics are needed to reliably predict bleeding and thrombotic risk in ambulatory left ventricular assist device (LVAD) patients. Hypothesis: Machine learning techniques can be used to predict risk of gastrointestinal bleeding (GIB), stroke, or death in ambulatory LVAD patients. Methods: HeartMate 3 TM LVAD recipients from the MOMENTUM 3 studies with up to 2-year follow-up were included. A multistate model was developed with 5-fold cross-validation to characterize the continuous probability of events after discharge. Model features included pre-implant, index implant, and short- and long-term post-implant clinical features. 85% of patients were used for model derivation and 15% for validation. Model performance was assessed with area under the curve (AUC). With the model features, a risk stratification tool was created by dividing patients’ into terciles of predicted risk. Results: Among 2,056 LVAD patients who survived to hospital discharge, the median age was 59.4 yrs (20.4% Female, 28.6% Black). At 2-years, the incidence of GIB, Stroke, and Death was 25.6%, 6.0% and 12.3%, respectively. Unique models including pre-implant, implant and post-implant features were created for GIB (Figure A), Stroke and Death. In ambulatory LVAD patients, the model could predict 30-day risk of GIB at any time post-implant which was 26.9%, 2.6% and 0.8% in high, medium and low-risk patients respectively (Figure B). Similar risk prediction was performed for stroke and death. The cross-validated AUC in the derivation cohort was 0.70, 0.69, and 0.85 for GIB, stroke, and death respectively. Conclusion: We developed an innovative risk tool informed by machine learning techniques that predicts risk of GIB, stroke, and death at any 30-day interval in ambulatory LVAD patients with readily available clinical data. The model allows for risk stratification that accurately predicts future events and may be useful to guide clinical decision making.

Baseline radiomics features and MYC rearrangement status predict progression in aggressive B-cell lymphoma

AbstractWe investigated whether the outcome prediction of patients with aggressive B-cell lymphoma can be improved by combining clinical, molecular genotype, and radiomics features. MYC, BCL2, and BCL6 rearrangements were assessed using fluorescence in situ hybridization. Seventeen radiomics features were extracted from the baseline positron emission tomography–computed tomography of 323 patients, which included maximum standardized uptake value (SUVmax), SUVpeak, SUVmean, metabolic tumor volume (MTV), total lesion glycolysis, and 12 dissemination features pertaining to distance, differences in uptake and volume between lesions, respectively. Logistic regression with backward feature selection was used to predict progression after 2 years. The predictive value of (1) International Prognostic Index (IPI); (2) IPI plus MYC; (3) IPI, MYC, and MTV; (4) radiomics; and (5) MYC plus radiomics models were tested using the cross-validated area under the curve (CV-AUC) and positive predictive values (PPVs). IPI yielded a CV-AUC of 0.65 ± 0.07 with a PPV of 29.6%. The IPI plus MYC model yielded a CV-AUC of 0.68 ± 0.08. IPI, MYC, and MTV yielded a CV-AUC of 0.74 ± 0.08. The highest model performance of the radiomics model was observed for MTV combined with the maximum distance between the largest lesion and another lesion, the maximum difference in SUVpeak between 2 lesions, and the sum of distances between all lesions, yielding an improved CV-AUC of 0.77 ± 0.07. The same radiomics features were retained when adding MYC (CV-AUC, 0.77 ± 0.07). PPV was highest for the MYC plus radiomics model (50.0%) and increased by 20% compared with the IPI (29.6%). Adding radiomics features improved model performance and PPV and can, therefore, aid in identifying poor prognosis patients.

External Validation and Addition of Prostate-specific Membrane Antigen Positron Emission Tomography to the Most Frequently Used Nomograms for the Prediction of Pelvic Lymph-node Metastases: an International Multicenter Study

BackgroundDifferent nomograms exist for the preoperative prediction of pelvic lymph-node metastatic disease in individual patients with prostate cancer (PCa). These nomograms do not incorporate modern imaging techniques such as prostate-specific membrane antigen (PSMA) positron emission tomography (PET). ObjectiveTo determine the predictive performance of the Briganti 2017, Memorial Sloan Kettering Cancer Center (MSKCC), and Briganti 2019 nomograms with the addition of PSMA-PET in an international, multicenter, present-day cohort of patients undergoing robot-assisted radical prostatectomy (RARP) and extended pelvic lymph-node dissection (ePLND) for localized PCa. Design, setting, and participantsAll 757 eligible patients who underwent a PSMA-PET prior to RARP and ePLND in three reference centers for PCa surgery between January 2016 and November 2020 were included. Outcome measurements and statistical analysisPerformance of the three nomograms was assessed using the receiver operating characteristic curve–derived area under the curve (AUC), calibration plots, and decision curve analyses. Subsequently, recalibration and addition of PSMA-PET to the nomograms were performed. Results and limitationsOverall, 186/757 patients (25%) had pelvic lymph-node metastatic (pN1) disease on histopathological examination. AUCs of the Briganti 2017, MSKCC, and Briganti 2019 nomograms were 0.70 (95% confidence interval [95% CI]: 0.64–0.77), 0.71 (95% CI: 0.65–0.77), and 0.76 (95% CI: 0.71–0.82), respectively. PSMA-PET findings showed a significant association with pN1 disease when added to the nomograms (p < 0.001). Addition of PSMA-PET substantially improved the discriminative ability of the models yielding cross-validated AUCs of 0.76 (95% CI: 0.70–0.82), 0.77 (95% CI: 0.72–0.83), and 0.82 (95% CI: 0.76–0.87), respectively. In decision curve analyses, the addition of PSMA-PET to the three nomograms resulted in increased net benefits. ConclusionsThe addition of PSMA-PET to the previously developed nomograms showed substantially improved predictive performance, which suggests that PSMA-PET is a likely future candidate for a modern predictive nomogram. Patient summaryDifferent tools have been developed to individualize the prediction of prostate cancer spread to lymph nodes before surgery. We found that the inclusion of modern imaging (prostate-specific membrane antigen positron emission tomography) improved substantially the overall performance of these prediction tools.

Diagnosing mild traumatic brain injury using saliva RNA compared to cognitive and balance testing.

BackgroundEarly, accurate diagnosis of mild traumatic brain injury (mTBI) can improve clinical outcomes for patients, but mTBI remains difficult to diagnose because of reliance on subjective symptom reports. An objective biomarker could increase diagnostic accuracy and improve clinical outcomes. The aim of this study was to assess the ability of salivary noncoding RNA (ncRNA) to serve as a diagnostic adjunct to current clinical tools. We hypothesized that saliva ncRNA levels would demonstrate comparable accuracy for identifying mTBI as measures of symptom burden, neurocognition, and balance.MethodsThis case‐control study involved 538 individuals. Participants included 251 individuals with mTBI, enrolled ≤14 days postinjury, from 11 clinical sites. Saliva samples (n = 679) were collected at five time points (≤3, 4‐7, 8‐14, 15‐30, and 31‐60 days post‐mTBI). Levels of ncRNAs (microRNAs, small nucleolar RNAs, and piwi‐interacting RNAs) were quantified within each sample using RNA sequencing. The first sample from each mTBI participant was compared to saliva samples from 287 controls. Samples were divided into testing (n = 430; mTBI = 201 and control = 239) and training sets (n = 108; mTBI = 50 and control = 58). The test set was used to identify ncRNA diagnostic candidates and create a diagnostic model. Model accuracy was assessed in the naïve test set.ResultsA model utilizing seven ncRNA ratios, along with participant age and chronic headache status, differentiated mTBI and control participants with a cross‐validated area under the curve (AUC) of .857 in the training set (95% CI, .816‐.903) and .823 in the naïve test set. In a subset of participants (n = 321; mTBI = 176 and control = 145) assessed for symptom burden (Post‐Concussion Symptom Scale), as well as neurocognition and balance (ClearEdge System), these clinical measures yielded cross‐validated AUC of .835 (95% CI, .782‐.880) and .853 (95% CI, .803‐.899), respectively. A model employing symptom burden and four neurocognitive measures identified mTBI participants with similar AUC (.888; CI, .845‐.925) as symptom burden and four ncRNAs (.932; 95% CI, .890‐.965).ConclusionSalivary ncRNA levels represent a noninvasive, biologic measure that can aid objective, accurate diagnosis of mTBI.

A Targeted Multiomics Approach to Identify Biomarkers Associated with Rapid eGFR Decline in Type 1 Diabetes

Background: Individuals with type 1 diabetes (T1D) demonstrate varied trajectories of estimated glomerular filtration rate (eGFR) decline. The molecular pathways underlying rapid eGFR decline in T1D are poorly understood, and individual-level risk of rapid eGFR decline is difficult to predict. Methods: We designed a case-control study with multiple exposure measurements nested within 4 well-characterized T1D cohorts (FinnDiane, Steno, EDC, and CACTI) to identify biomarkers associated with rapid eGFR decline. Here, we report the rationale for and design of these studies as well as results of models testing associations of clinical characteristics with rapid eGFR decline in the study population, upon which “omics” studies will be built. Cases (n = 535) and controls (n = 895) were defined as having an annual eGFR decline of ≥3 and <1 mL/min/1.73 m², respectively. Associations of demographic and clinical variables with rapid eGFR decline were tested using logistic regression, and prediction was evaluated using area under the curve (AUC) statistics. Targeted metabolomics, lipidomics, and proteomics are being performed using high-resolution mass-spectrometry techniques. Results: At baseline, the mean age was 43 years, diabetes duration was 27 years, eGFR was 94 mL/min/1.73 m², and 62% of participants were normoalbuminuric. Over 7.6-year median follow-up, the mean annual change in eGFR in cases and controls was −5.7 and 0.6 mL/min/1.73 m², respectively. Younger age, longer diabetes duration, and higher baseline HbA1c, urine albumin-creatinine ratio, and eGFR were significantly associated with rapid eGFR decline. The cross-validated AUC for the predictive model incorporating these variables plus sex and mean arterial blood pressure was 0.74 (95% CI: 0.68–0.79; p < 0.001). Conclusion: Known risk factors provide moderate discrimination of rapid eGFR decline. Identification of blood and urine biomarkers associated with rapid eGFR decline in T1D using targeted omics strategies may provide insight into disease mechanisms and improve upon clinical predictive models using traditional risk factors.

Eye Tracking as a Biomarker for Concussion in Children.

Concussion is the most common type of brain injury in both pediatric and adult populations and can potentially result in persistent postconcussion symptoms. Objective assessment of physiologic "mild" traumatic brain injury in concussion patients remains challenging. This study evaluates an automated eye-tracking algorithm as a biomarker for concussion as defined by its symptoms and the clinical signs of convergence insufficiency and accommodation dysfunction in a pediatric population. Cross-sectional case-control study. Primary care. Concussed children (N = 56; mean age = 13 years), evaluated at a mean of 22-week post-injury, compared with 83 uninjured controls. Metrics comparing velocity and conjugacy of eye movements over time were obtained and were compared with the correlation between Acute Concussion Evaluation (ACE) scores, convergence, and accommodation dysfunction. Subjects' eye movements recorded with an automated eye tracker while they watched a 220-second cartoon film clip played continuously while moving within an aperture. Twelve eye-tracking metrics were significantly different between concussed and nonconcussed children. A model to classify concussion as diagnosed by its symptoms assessed using the ACE achieved an area under the curve (AUC) = 0.854 (71.9% sensitivity, 84.4% specificity, a cross-validated AUC = 0.789). An eye-tracking model built to identify near point of convergence (NPC) disability achieved 95.8% specificity and 57.1% sensitivity for an AUC = 0.810. Reduced binocular amplitude of accommodation had a Spearman correlation of 0.752(P value <0.001) with NPC. Eye tracking correlated with concussion symptoms and detected convergence and accommodative abnormalities associated with concussion in the pediatric population. It demonstrates utility as a rapid, objective, noninvasive aid in the diagnosis of concussion.

Predictive Value of Breast MRI Background Parenchymal Enhancement for Neoadjuvant Treatment Response among HER2- Patients.

Women with advanced HER2- breast cancer have limited treatment options. Breast MRI functional tumor volume (FTV) is used to predict pathologic complete response (pCR) to improve treatment efficacy. In addition to FTV, background parenchymal enhancement (BPE) may predict response and was explored for HER2- patients in the I-SPY-2 TRIAL. Women with HER2- stage II or III breast cancer underwent prospective serial breast MRIs during four neoadjuvant chemotherapy timepoints. BPE was quantitatively calculated using whole-breast manual segmentation. Logistic regression models were systematically explored using pre-specified and optimized predictor selection based on BPE or combined with FTV. A total of 352 MRI examinations in 88 patients (29 with pCR, 59 non-pCR) were evaluated. Women with hormone receptor (HR)+HER2- cancers who achieved pCR demonstrated a significantly greater decrease in BPE from baseline to pre-surgery compared to non-pCR patients (odds ratio 0.64, 95% confidence interval (CI): 0.39-0.92, P = 0.04). The associated BPE area under the curve (AUC) was 0.77 (95% CI: 0.56-0.98), comparable to the range of FTV AUC estimates. Among multi-predictor models, the highest cross-validated AUC of 0.81 (95% CI: 0.73-0.90) was achieved with combined FTV+HR predictors, while adding BPE to FTV+HR models had an estimated AUC of 0.82 (95% CI: 0.74-0.92). Among women with HER2- cancer, BPE alone demonstrated association with pCR in women with HR+HER2- breast cancer, with similar diagnostic performance to FTV. BPE predictors remained significant in multivariate FTV models, but without added discrimination for pCR prediction. This may be due to small sample size limiting ability to create subtype-specific multivariate models.

Bruch's Membrane Opening Minimum Rim Width Provides Objective Differentiation between Glaucoma and Nonglaucomatous Optic Neuropathies

A challenging clinical scenario is distinguishing between normal tension glaucoma (NTG) and non-glaucomatous optic neuropathies (NGON). The key to the assessment remains identifying the presence of optic nerve head cupping. Recent optical coherence tomography (OCT) measurements now allow objective assessment of cupping by minimum rim width at Bruch's membrane opening (MRW-BMO). This study assessed the hypothesis that the MRW-BMO measurement quantifies cupping and therefore can differentiate between NTG and NGON. Diagnostic evaluation with area under the curve. Setting: multicenter tertiary hospitals and outpatient clinics. 81 eyes of 81 patients were enrolled, 27 with NTG and 54 with NGON, including ischemic optic neuropathy, previous optic neuritis, and compressive and inherited optic neuropathies. All NGON patients with intraocular pressure >21mm Hg, narrow drainage angles, or a family history of glaucoma were excluded. Observational procedure: optic disc OCT images were obtained of both the retinal nerve fiber layer thickness and the MRW-BMO. the utility of the MRW-BMO in differentiating GON from NGON was assessed using the area under the curve (AUC) estimated from a logistic regression model. The 5-fold cross-validated AUC for glaucoma versus nonglaucoma from logistic regression models using MRW-BMO values from all sectors was 0.95 (95% confidence interval: 0.86-1.00). The measurement of MRW-BMO effectively differentiates between NTG and NGON with a high level of sensitivity and specificity. Incorporating this measurement into routine glaucoma assessment may provide a robust method of assisting clinicians to improve diagnosis and therefore treatment of optic nerve diseases.

The first composite score predicting Digital Ulcers in systemic sclerosis patients using Clinical data, Imaging and Patient history\u2014CIP-DUS

BackgroundDigital ulcers (DU) present a challenging complication in systemic sclerosis (SSc). The aim of this study was to combine clinical characteristics and imaging methods to a composite score for the prediction of DU in SSc patients.MethodsSeventy-nine SSc patients received clinical examination, their patient history was taken and nailfold capillaroscopy (NC), colour Doppler ultrasonography (CDUS) and fluorescence optical imaging (FOI) of the hands were performed at baseline. Newly developed DU over a period of approximately 12 months were registered. We used criteria with area under the curve (AUC) of at least 0.6 in regard to the development of these new DU to create the score (CIP-DUS, clinical features, imaging, patient history—digital ulcer score).ResultsTwenty-nine percent of all SSc patients developed new DU during follow-up (48.1% diffuse, 18.4% limited SSc). Based on the cross-validated (cv) AUC, a weight (cvAUC > 0.6 and ≤ 0.65: 1; cvAUC > 0.65 and ≤ 0.7: 2; cvAUC > 0.7: 3) was assigned to each selected parameter. The performance of the final CIP-DUS was assessed with and without the CDUS/FOI component. For the scleroderma patterns in NC, three points were appointed to late, two to active and one point to early capillaroscopy pattern according to Cutolo et al. The CIP-DUS including the CDUS and FOI parameters resulted in a good diagnostic performance (AUC after cross-validation: 0.83, 95%CI 0.74 to 0.92) and was well calibrated (chi-square = 12.3, p = 0.58). The cut-off associated with the maximum of sensitivity and specificity was estimated at ≥ 10 points resulting in a sensitivity of 100% and specificity of 74% for new DU during follow-up. Excluding CDUS and FOI parameters leads to a non-statistically significant lower performance (AUC after cross-validation: 0.81, 95%CI 0.72 to 0.91). However, including CDUS and FOI resulted in a better classification of patients in respect to the outcome new DU in follow-up due to significantly better reclassification performance (NRI = 62.1, p = 0.001) and discrimination improvement (IDI = 9.7, p = 0.01).ConclusionA new score was introduced with the aim to predict digital ulcers. If applied correctly and with the new imaging techniques proposed, all patients at risk of digital ulcers throughout 12 months could be identified.

Non-motor Clinical and Biomarker Predictors Enable High Cross-Validated Accuracy Detection of Early PD but Lesser Cross-Validated Accuracy Detection of Scans Without Evidence of Dopaminergic Deficit.

Background: Early stage (preclinical) detection of Parkinson's disease (PD) remains challenged yet is crucial to both differentiate it from other disorders and facilitate timely administration of neuroprotective treatment as it becomes available.Objective: In a cross-validation paradigm, this work focused on two binary predictive probability analyses: classification of early PD vs. controls and classification of early PD vs. SWEDD (scans without evidence of dopamine deficit). It was hypothesized that five distinct model types using combined non-motor and biomarker features would distinguish early PD from controls with > 80% cross-validated (CV) accuracy, but that the diverse nature of the SWEDD category would reduce early PD vs. SWEDD CV classification accuracy and alter model-based feature selection.Methods: Cross-sectional, baseline data was acquired from the Parkinson's Progressive Markers Initiative (PPMI). Logistic regression, general additive (GAM), decision tree, random forest and XGBoost models were fitted using non-motor clinical and biomarker features. Randomized train and test data partitions were created. Model classification CV performance was compared using the area under the curve (AUC), sensitivity, specificity and the Kappa statistic.Results: All five models achieved >0.80 AUC CV accuracy to distinguish early PD from controls. The GAM (CV AUC 0.928, sensitivity 0.898, specificity 0.897) and XGBoost (CV AUC 0.923, sensitivity 0.875, specificity 0.897) models were the top classifiers. Performance across all models was consistently lower in the early PD/SWEDD analyses, where the highest performing models were XGBoost (CV AUC 0.863, sensitivity 0.905, specificity 0.748) and random forest (CV AUC 0.822, sensitivity 0.809, specificity 0.721). XGBoost detection of non-PD SWEDD matched 1–2 years curated diagnoses in 81.25% (13/16) cases. In both early PD/control and early PD/SWEDD analyses, and across all models, hyposmia was the single most important feature to classification; rapid eye movement behavior disorder (questionnaire) was the next most commonly high ranked feature. Alpha-synuclein was a feature of import to early PD/control but not early PD/SWEDD classification and the Epworth Sleepiness scale was antithetically important to the latter but not former.Interpretation: Non-motor clinical and biomarker variables enable high CV discrimination of early PD vs. controls but are less effective discriminating early PD from SWEDD.

PD01-08 PREDICTING RESIDUAL STONE FRAGMENTS IN PATIENTS UNDERGOING URETEROSCOPY IS DIFFICULT DESPITE SIGNIFICANT ASSOCIATIONS BETWEEN PREDICTORS AND OUTCOME: RESULTS FROM THE MUSIC STATEWIDE COLLABORATIVE

You have accessJournal of UrologyStone Disease: Surgical Therapy I (PD01)1 Apr 2020PD01-08 PREDICTING RESIDUAL STONE FRAGMENTS IN PATIENTS UNDERGOING URETEROSCOPY IS DIFFICULT DESPITE SIGNIFICANT ASSOCIATIONS BETWEEN PREDICTORS AND OUTCOME: RESULTS FROM THE MUSIC STATEWIDE COLLABORATIVE Richard Wu*, Adharsh Murali, Colton Walker, Casey Dauw, Kavya Swarna, Arvin George, Khurshid Ghani, Jill Slayton, Karandeep Singh, and for the Michigan Urological Surgery Improvement Collaborative Richard Wu*Richard Wu* More articles by this author , Adharsh MuraliAdharsh Murali More articles by this author , Colton WalkerColton Walker More articles by this author , Casey DauwCasey Dauw More articles by this author , Kavya SwarnaKavya Swarna More articles by this author , Arvin GeorgeArvin George More articles by this author , Khurshid GhaniKhurshid Ghani More articles by this author , Jill SlaytonJill Slayton More articles by this author , Karandeep SinghKarandeep Singh More articles by this author , and for the Michigan Urological Surgery Improvement Collaborative More articles by this author View All Author Informationhttps://doi.org/10.1097/JU.0000000000000821.08AboutPDF ToolsAdd to favoritesDownload CitationsTrack CitationsPermissionsReprints ShareFacebookLinked InTwitterEmail Abstract INTRODUCTION AND OBJECTIVE: Although one of the measures of successful ureteroscopy (URS) for stone disease is the absence of residual stone fragments, limited evidence is available on whether it is possible to predict which patients will have residual fragments following URS. Availability of a risk model would help identify patients who may benefit from follow-up imaging and additional procedures. We sought to determine whether residual stone fragments following URS can be predicted using data from a large prospective clinical registry. METHODS: The Michigan Urological Surgery Improvement Collaborative (MUSIC) is a consortium of 46 diverse urology practices, of which 40 have participated in a registry of patients with kidney and ureteral stones since 2016. Among patients who underwent URS with available pre and post URS imaging, we developed univariable and multivariable logistic regression models to predict stone-free status using demographic and stone-related factors, including age, sex, body mass index, presence of diabetes, size of largest stone visualized, location of stone(s), number of stones, and presence of pre-procedure hydronephrosis and urinary tract infection. We assessed model discrimination using 5 fold cross-validated area-under-the-curve (AUC). RESULTS: We identified 3,971 patients who underwent URS with available pre- and post-URS imaging, of whom 1,700 (42.8%) had residual stone fragments. Variables with statistically significant associations with residual stone fragments in both univariable and multivariable models included age, body mass index, stone size, several stone locations, and number of kidney stones (Table 1). Cross-validated AUC for the model was 0.635.Table 1. Odds ratios for univariable and multivariable analysis for predicting residual stone fragments following URS. CONCLUSIONS: Despite statistically significant associations between several predictors and the outcome, a logistic regression model predicting presence of post-URS residual stone fragments had poor discrimation. Source of Funding: Blue Cross Blue Shield of Michigan © 2020 by American Urological Association Education and Research, Inc.FiguresReferencesRelatedDetails Volume 203Issue Supplement 4April 2020Page: e62-e63 Advertisement Copyright & Permissions© 2020 by American Urological Association Education and Research, Inc.MetricsAuthor Information Richard Wu* More articles by this author Adharsh Murali More articles by this author Colton Walker More articles by this author Casey Dauw More articles by this author Kavya Swarna More articles by this author Arvin George More articles by this author Khurshid Ghani More articles by this author Jill Slayton More articles by this author Karandeep Singh More articles by this author for the Michigan Urological Surgery Improvement Collaborative More articles by this author Expand All Advertisement PDF downloadLoading ...

Abstract PD9-06: Additive benefit of MRI background parenchymal enhancement (BPE) to an MRI tumor volume model for predicting pathologic complete response (pCR) to neoadjuvant treatment in ISPY2

Abstract BACKGROUND: Background parenchymal enhancement (BPE) describes the natural phenomenon observed on breast MRI in which normal breast tissue demonstrates signal enhancement from uptake of intravenous contrast. BPE may provide independent and additive value for prediction of pathologic complete response (pCR) using MRI measured functional tumor volume (FTV), which has only moderate discrimination (FTV AUC ~ 0.7). We evaluated the additive value of quantitative whole breast BPE to a FTV model for prediction of pCR to neoadjuvant chemotherapy in the ISPY-2 trial. METHODS: In this HIPAA-compliant/IRB-approved study, women 18 years of age and older diagnosed with stage II or III breast cancer and with tumor size measured ≥ 2.5 cm were eligible to enroll in the I-SPY 2 TRIAL. Participants received a weekly dose of paclitaxel alone (control) or in combination with Veliparib and Carboplatin for 12 weekly cycles followed by four (every 2-3 weeks) cycles of anthracycline-cyclophosphamide prior to surgery. All breast cancers in these drug arms were Her2 negative. MRI was performed before the initiation of neoadjuvant therapy or “baseline” (T0), after three weeks of therapy or “early treatment” (T1), after twelve weeks between drug regimens or “inter-regimen” (T2), and after neoadjuvant therapy completion and prior to surgery or “pre-surgery” (T3). MRI segmentation was manually performed of the whole unaffected contralateral breast, and tissue classification was performed using fuzzy c-means clustering. BPE was calculated as the average enhancement of all tissue voxels at the first postcontrast acquisition. Predictor variables were parameterized as absolute values of BPE/FTV for each treatment time point or relative change values for each treatment time period. Logistic regression, stratified by hormone receptor (HR) subtype, was performed using 1) univariate models of FTV/BPE predictors alone and 2) multivariate models using all possible combinations of FTV/BPE predictors and HR status. Additive benefit for multivariate models was evaluated by estimating change in the area under the curve (AUC) for overall diagnostic performance with 10-repeat 5-fold cross validation. The 95% confidence interval (CI) of cross-validated AUC was estimated using 1,000 bootstrap resamples. RESULTS: A total of 88 patients (29 pCR, 59 non-pCR) were evaluated with serial breast MRIs to assess neoadjuvant response. Among univariate models, women with HR+ cancers who had PCR demonstrated a significantly greater decrease in BPE from baseline to pre-surgery compared to non-PCR (OR = 0.64, 95% CI = 0.39-0.92, p-value = 0.04). The associated AUC was 0.77 (95% CI 0.56-0.98), comparable to the range of univariate FTV AUC values (0.57-0.87). Among optimized multivariate models, the highest cross-validated AUC for FTV and HR predictors was 0.81 (95% CI 0.73-0.90), while adding BPE slightly increased AUC to 0.82 (95% CI 0.74-0.92). CONCLUSION: Changes in BPE in response to neoadjuvant therapy, which represents normal breast tissue changes measurable on any breast MRI, demonstrated significant association with pCR in women with HR+ breast cancer. Moreover, it had a similar diagnostic performance to univariate prediction with tumor volume. However, additive prediction of BPE to multivariate FTV models was only marginal. Citation Format: Vignesh A. Arasu, Paul Kim, Wen Li, Fredrik Strand, Cody McHargue, Ella Jones, David Newitt, John Kornak, Laura Esserman, Nola Hylton, ACRIN 6657 Trial TeamI-SPY Investigators Network. Additive benefit of MRI background parenchymal enhancement (BPE) to an MRI tumor volume model for predicting pathologic complete response (pCR) to neoadjuvant treatment in ISPY2 [abstract]. In: Proceedings of the 2019 San Antonio Breast Cancer Symposium; 2019 Dec 10-14; San Antonio, TX. Philadelphia (PA): AACR; Cancer Res 2020;80(4 Suppl):Abstract nr PD9-06.

A Monte Carlo based radiation response modelling framework to assess variability of clinical RBE in proton therapy

The clinical implementation of a variable relative biological effectiveness (RBE) in proton therapy is currently controversially discussed. Initial clinical evidence indicates a variable proton RBE, which needs to be verified. In this study, a radiation response modelling framework for assessing clinical RBE variability is established. It was applied to four selected glioma patients (grade III) treated with adjuvant radio(chemo)therapy and who developed late morphological image changes on T1-weighted contrast-enhanced (T1w-CE) magnetic resonance (MR) images within approximately two years of recurrence-free follow-up. The image changes were correlated voxelwise with dose and linear energy transfer (LET) values using univariable and multivariable logistic regression analysis. The regression models were evaluated by the area-under-the-curve (AUC) method performing a leave-one-out cross validation. The tolerance dose TD50 at which 50% of patient voxels experienced toxicity was interpolated from the models. A Monte Carlo (MC) model was developed to simulate dose and LET distributions, which includes variance reduction (VR) techniques to decrease computation time. Its reliability and accuracy were evaluated based on dose calculations of the clinical treatment planning system (TPS) as well as absolute dose measurements performed in the patient specific quality assurance.Morphological image changes were related to a combination of dose and LET. The multivariable models revealed cross-validated AUC values of up to 0.88. The interpolated TD50 curves decreased with increasing LET indicating an increase in biological effectiveness. The MC model reliably predicted average TPS dose within the clinical target volume as well as absolute water phantom dose measurements within 2% accuracy using dedicated VR settings.The observed correlation of dose and LET with late brain tissue damage suggests considering RBE variability for predicting chronic radiation-induced brain toxicities. The MC model simulates radiation fields in patients precisely and time-efficiently. Hence, this study encourages and enables in-depth patient evaluation to assess the variability of clinical proton RBE.

Cvauroc: Command to compute cross-validated area under the curve for ROC analysis after predictive modeling for binary outcomes

Receiver operating characteristic (ROC) analysis is used for comparing predictive models in both model selection and model evaluation. ROC analysis is often applied in clinical medicine and social science to assess the tradeoff between model sensitivity and specificity. After fitting a binary logistic or probit regression model with a set of independent variables, the predictive performance of this set of variables can be assessed by the area under the curve (AUC) from an ROC curve. An important aspect of predictive modeling (regardless of model type) is the ability of a model to generalize to new cases. Evaluating the predictive performance (AUC) of a set of independent variables using all cases from the original analysis sample often results in an overly optimistic estimate of predictive performance. One can use K-fold cross-validation to generate a more realistic estimate of predictive performance in situations with a small number of observations. AUC is estimated iteratively for k samples (the “test” samples) that are independent of the sample used to predict the dependent variable (the “training” sample). cvauroc implements k-fold cross-validation for the AUC for a binary outcome after fitting a logit or probit regression model, averaging the AUCs corresponding to each fold, and bootstrapping the cross-validated AUC to obtain statistical inference and 95% confidence intervals. Furthermore, cvauroc optionally provides the cross-validated fitted probabilities for the dependent variable or outcome, contained in a new variable named _fit; the sensitivity and specificity for each of the levels of the predicted outcome, contained in two new variables named _sen and _spe; and the plot of the mean cross-validated AUC and k-fold ROC curves.

The Predictive Performance of Objective Measures of Physical Activity Derived From Accelerometry Data for 5-Year All-Cause Mortality in Older Adults: National Health and Nutritional Examination Survey 2003-2006.

Declining physical activity (PA) is a hallmark of aging. Wearable technology provides reliable measures of the frequency, duration, intensity, and timing of PA. Accelerometry-derived measures of PA are compared with established predictors of 5-year all-cause mortality in older adults in terms of individual, relative, and combined predictive performance. Participants aged between 50 and 85 years from the 2003-2006 National Health and Nutritional Examination Survey (NHANES, n = 2,978) wore a hip-worn accelerometer in the free-living environment for up to 7 days. A total of 33 predictors of 5-year all-cause mortality (number of events = 297), including 20 measures of objective PA, were compared using univariate and multivariate logistic regression. In univariate logistic regression, the total activity count was the best predictor of 5-year mortality (Area under the Curve (AUC) = 0.771) followed by age (AUC = 0.758). Overall, 9 of the top 10 predictors were objective PA measures (AUC from 0.771 to 0.692). In multivariate regression, the 10-fold cross-validated AUC was 0.798 for the model without objective PA variables (9 predictors) and 0.838 for the forward selection model with objective PA variables (13 predictors). The Net Reclassification Index was substantially improved by adding objective PA variables (p < .001). Objective accelerometry-derived PA measures outperform traditional predictors of 5-year mortality, including age. This highlights the importance of wearable technology for providing reproducible, unbiased, and prognostic biomarkers of health.

A Signature of Gene Expression in Peripheral Blood that Enables Earlier Detection of Acute Rejection in Kidney Transplant Recipients

Background Acute rejection (AR) is often indicated by an increase in creatinine levels, after a considerable graft damage has already occurred. There is, therefore, a need for early identification of AR. Monitoring of RNA gene signatures in peripheral blood and urine samples offers the opportunity for surveillance of the recipient immune system and earlier detection of acute rejection. Samples We examined RT-qPCR expression of 22 literature-based candidate genes in 335 longitudinal whole-blood samples from 27 stable kidney transplant recipients (KTRs) from the KALIBRE study (Kidney Allograft Immune Biomarkers of Rejection Episodes) and 237 samples from 27 KTRs with a biopsy-proven cellular rejection during the first post-transplant year. Creatinine in stable KTRs reached no higher than 20% above the reference range and had less than 15% standard deviation. Methods To account for post-transplantation changes unrelated to rejection, we generated time-adjusted gene expression as the residuals of linear mixed-effects models with stable patients. To select a parsimonious AR signature, we used penalised logistic regression with a pre-biopsy sample in rejectors and the median gene expression for each stable patient. As a test set we used the longitudinal data. We were also able to examine longitudinal expression of genes as well as effect of different anti-rejection therapies. We subsequently tested the performance of our signature in a second cohort. Results A 7-gene signature of AR showed area under the curve (AUC) 0.95 (95% confidence interval 0.91-1.00), sensitivity 0.85, specificity 0.93, positive and negative predictive values 0.92 and 0.86 and cross-validated AUC 0.85 (0.75-0.95). The estimated probability of rejection increased as early as six weeks prior to the AR biopsy and decreased after administration of treatment. Notably, gene expression in stable KTRs, as well as in rejectors, showed a pronounced variability, with 22% AR-positives longitudinal samples.Conclusion Molecular marker alterations in blood emerge well ahead of the time of clinically overt AR. Monitoring of a gene-expression signature of AR in peripheral blood could unravel AR-related pro-inflammatory activity and provide information on subclinical processes and further justification for a biopsy in cases of sustained kidney dysfunction with no marked change in creatinine. Medical Research Council (MRC) (grants G0801537/ID: 88245 and MRC Centre for Transplantation, – MRC grant no. MR/J006742/1). MHF - HEALTH-F5– 2010–260687: The ONE Study. SC, PM and MPH-F received funding from FP7-HEALTH-2012-INNOVATION-1 project number 305147: BIO-DrIM. National Institute for Health Research Biomedical Research Centre at Guy’s and St Thomas’ and King’s College London. NIHR Health Informatics Collaborative.

A classification model for predicting diabetic retinopathy based on patient characteristics and biochemical measures

Purpose: In the United Kingdom (UK), The NHS Diabetic Eye Screening Program offers an annual eye examination to all people with diabetes aged 12 or over, aiming at the early detection of people at high risk of visual loss due to diabetic retinopathy. The purpose of this study was the design of a model to predict patients at risk of developing retinopathy with the use of patient characteristics and clinical measures. Methods: We investigated data from 2011 to 2016 from the population-based Diabetic Eye Screening Program in East Anglia. The data comprised retinal eye screening results, patient characteristics, and routine biochemical measures of HbA1c, blood pressure, Albumin to Creatinine ratio (ACR), estimated Glomerular Filtration rate (eGFR), serum creatinine, cholesterol and Body Mass Index (BMI). Individuals were classified according to the presence or absence of retinopathy as indicated by their retinal eye examinations. A lasso regression, random forest, gradient boosting machine and regularized gradient boosting model were built and cross-validated for their predictive ability. Results: A total of 6,375 subjects with recorded information for all available biochemical measures were identified from the cohorts. Of these, 5,969 individuals had no signs of diabetic retinopathy. Of the remainder 406 individuals with signs of diabetic retinopathy, 352 had background diabetic retinopathy and 54 had referable diabetic retinopathy. The highest value of the10-fold cross-validated Area under the Curve (AUC) was achieved by the gradient boosting machine 0.73 ± 0.03 and the minimum required set of variables to yield this performance included 4 variables: duration of diabetes, HbA1c, ACR and age. A subsequent analysis on the predictive power of the biochemical measures showed that when HbA1c and ACR measurements were available for longer time periods, the performance of the models was greatly enhanced. When HbA1c and ACR measurements for a 5-year period prior to the event of study were available, gradient boosting machine cross-validated AUC was 0.77 ± 0.04 in comparison to the cross-validated AUC of 0.68 ± 0.04 when only information for the 1-year period for these variables was available. Similarly, an increment from 0.70 ± 0.02 to 0.75 ± 0.04 was observed with random forest. The dataset with the 1-year measurements comprised 4,857 subjects, of whom, 4,572 had no retinopathy and the remainder 285 had signs of retinopathy. The dataset with the 5-year measurements comprised 757 subjects, of whom, 696 had no retinopathy and the remainder 51 had signs of retinopathy. Conclusions: The utilization of patient information and routine biochemical measures can be used to identify patients at risk of developing retinopathy. The effective differentiation between patients with and without retinopathy could significantly reduce the number of screening visits without compromising patients’ health.

Construction of a pathological risk model of occult lymph node metastases for prognostication by semi-automated image analysis of tumor budding in early-stage oral squamous cell carcinoma

It is challenging to identify at diagnosis those patients with early oral squamous cell carcinoma (OSCC), who have a poor prognosis and those that have a high risk of harboring occult lymph node metastases. The aim of this study was to develop a standardized and objective digital scoring method to evaluate the predictive value of tumor budding. We developed a semi-automated image-analysis algorithm, Digital Tumor Bud Count (DTBC), to evaluate tumor budding. The algorithm was tested in 222 consecutive patients with early-stage OSCC and major endpoints were overall (OS) and progression free survival (PFS). We subsequently constructed and cross-validated a binary logistic regression model and evaluated its clinical utility by decision curve analysis. A high DTBC was an independent predictor of both poor OS and PFS in a multivariate Cox regression model. The logistic regression model was able to identify patients with occult lymph node metastases with an area under the curve (AUC) of 0.83 (95% CI: 0.78–0.89, P <0.001) and a 10-fold cross-validated AUC of 0.79. Compared to other known histopathological risk factors, the DTBC had a higher diagnostic accuracy. The proposed, novel risk model could be used as a guide to identify patients who would benefit from an up-front neck dissection.

Modeling Reactivity to Biological Macromolecules with a Deep Multitask Network.

Most small-molecule drug candidates fail before entering the market, frequently because of unexpected toxicity. Often, toxicity is detected only late in drug development, because many types of toxicities, especially idiosyncratic adverse drug reactions (IADRs), are particularly hard to predict and detect. Moreover, drug-induced liver injury (DILI) is the most frequent reason drugs are withdrawn from the market and causes 50% of acute liver failure cases in the United States. A common mechanism often underlies many types of drug toxicities, including both DILI and IADRs. Drugs are bioactivated by drug-metabolizing enzymes into reactive metabolites, which then conjugate to sites in proteins or DNA to form adducts. DNA adducts are often mutagenic and may alter the reading and copying of genes and their regulatory elements, causing gene dysregulation and even triggering cancer. Similarly, protein adducts can disrupt their normal biological functions and induce harmful immune responses. Unfortunately, reactive metabolites are not reliably detected by experiments, and it is also expensive to test drug candidates for potential to form DNA or protein adducts during the early stages of drug development. In contrast, computational methods have the potential to quickly screen for covalent binding potential, thereby flagging problematic molecules and reducing the total number of necessary experiments. Here, we train a deep convolution neural network—the XenoSite reactivity model—using literature data to accurately predict both sites and probability of reactivity for molecules with glutathione, cyanide, protein, and DNA. On the site level, cross-validated predictions had area under the curve (AUC) performances of 89.8% for DNA and 94.4% for protein. Furthermore, the model separated molecules electrophilically reactive with DNA and protein from nonreactive molecules with cross-validated AUC performances of 78.7% and 79.8%, respectively. On both the site- and molecule-level, the model’s performances significantly outperformed reactivity indices derived from quantum simulations that are reported in the literature. Moreover, we developed and applied a selectivity score to assess preferential reactions with the macromolecules as opposed to the common screening traps. For the entire data set of 2803 molecules, this approach yielded totals of 257 (9.2%) and 227 (8.1%) molecules predicted to be reactive only with DNA and protein, respectively, and hence those that would be missed by standard reactivity screening experiments. Site of reactivity data is an underutilized resource that can be used to not only predict if molecules are reactive, but also show where they might be modified to reduce toxicity while retaining efficacy. The XenoSite reactivity model is available at http://swami.wustl.edu/xenosite/p/reactivity.