Area Under The Receiver Operator Characteristic Curves Research Articles

Abstract PO5-13-03: Early Detection of Breast Cancer using Targeted Plasma Metabolomic Profiling

Abstract Background: Breast cancer (BC) is the second leading cause of cancer death among women. Accordingly, early diagnosis is key to the successful treatment, management, and care of BC. Recent studies confirmed that plasma metabolites could be reliable cancer biomarkers, allowing for the development of a minimally invasive routine blood test that can be used for screening, as well as for monitoring disease evolution in patients. The exact metabolic pathways involved in early BC development remain unclear. Metabolomic profiling of women with BC may help to identify new biomarkers to predict breast cancer long before symptoms appear. The purpose of this study was to validate a plasma metabolomic biomarker panel for an improved risk assessment for early detection BC in 241 patients, and to understand the potential role and the relationship between BC subtypes and hormone receptor status. Methods: Our study included a total of 185 plasma samples from women with biopsy-confirmed BC and 56 plasma samples from healthy controls. A targeted, quantitative mass spectrometry (MS)-based metabolomics approach was used to analyze 138 metabolites in plasma samples using a combination of direct injection (DI) MS and reverse-phase high performance liquid chromatography (HPLC) tandem mass spectrometry (MS/MS). The sample set was split into a discovery set and validation set. Metabolite concentration data, clinical data, and hormones receptor status were used to determine optimal biomarker sets. The same biomarkers and regression models were used and assessed on the validation models. The area under the receiver operator characteristic curves (AUROC), sensitivities and specificities at selected cut off points were calculated for each subgroup. Results: A large proportion of BC patients were at an early stage, with 98 at stage I (53.0 %), 70 at stage II (37.8%), and 17 at stage III (9.2%). The BC patients were of all subtypes: 138 luminal A (76,2%), 23 luminal B (12.7%), 5 hormone receptor-negative/HER2-positive (2.8%), and 15 triples negative (8.3%). A feature selection was performed on the training set using Partial Least-Squares Discriminant Analysis (PLS-DA), and the top performing metabolites were identified as the most important in discriminating BC from healthy subjects. The impact of age was also investigated. Features with >80% missing values were removed. To predict BC, the best signature comprised 9 variables implicated in fatty acid metabolism, amino acid metabolism, polyamine biosynthesis and related signaling pathways We further developed and validated a logistic regression with AUROC > 0.9 using these metabolites and other clinical data for detecting different stages and subtypes of BC. Conclusion: This study identified and validated a simple, high-performing, metabolite-based test for the early detection of BC. After confirmation in other independent cohort studies, our findings could provide the foundations for the development of a blood-based routine test for women at the highest risk for BC that is cost-effective, accurate and reliable. In addition, this approach could be used to complement other modalities especially for BC patients with dense breasts. Citation Format: Jean-Francois Haince, Lun Zhang, Rashid Ahmed Bux, Paramjit S. Tappia, Bram Ramjiawan, David Wishart, Andrew Maksymiuk. Early Detection of Breast Cancer using Targeted Plasma Metabolomic Profiling [abstract]. In: Proceedings of the 2023 San Antonio Breast Cancer Symposium; 2023 Dec 5-9; San Antonio, TX. Philadelphia (PA): AACR; Cancer Res 2024;84(9 Suppl):Abstract nr PO5-13-03.

The Diagnostic Accuracy of Procalcitonin, Soluble Urokinase-Type Plasminogen Activator Receptors, and C-Reactive Protein in Diagnosing Urinary Tract Infections in the Emergency Department-A Diagnostic Accuracy Study.

Background: Urinary tract infections (UTIs) are a leading bacterial infection in the emergency department (ED). Diagnosing UTIs in the ED can be challenging due to the heterogeneous presentation; therefore, fast and precise tests are needed. We aimed to evaluate the diagnostic precision of procalcitonin (PCT), soluble urokinase plasminogen activator receptors (suPARs), and C-reactive protein (CRP) in diagnosing UTIs, grading the severity of UTIs, and ruling out bacteremia. Methods: We recruited adults admitted to three Danish EDs with suspected UTIs. PCT, suPAR, and CRP were used in index tests, while blood cultures, expert panel diagnosis, and severity grading were used in the reference tests. Logistic regression and area under the receiver operator characteristic curves (AUROCs) were utilized to evaluate the models and determine the optimal cut-offs. Results: We enrolled 229 patients. PCT diagnosed UTI with an AUROC of 0.612, detected severe disease with an AUROC of 0.712, and ruled out bacteremia with an AUROC of 0.777. SuPAR had AUROCs of 0.480, 0.638, and 0.605, while CRP had AUROCs of 0.599, 0.778, and 0.646. Conclusions: The diagnostic performance of PCT, suPAR, or CRP for UTIs or to rule out severe disease was poor. However, PCT can safely rule out bacteremia in clinically relevant numbers in ED patients suspected of UTI.

O57: Tailored risk assessment and forecasting in intermittent claudication using machine learning (TRAFIC-ML)

Abstract Introduction Guidelines recommend cardiovascular risk reduction and supervised exercise therapy as the initial treatment for patients with intermittent claudication(IC). However, implementation challenges and poor patient compliance lead to significant variation in management and therefore outcomes. We propose a precise machine learning derived decision support system that aims to provide personalised outcome predictions across different management strategies. Methods Feature selection was performed using the least absolute shrinkage and selection operator method. The model was developed using a bootstrapped sample of 10,000 patients based on 255 patients from our vascular centre. The model considered 27 baseline characteristics, compliance to best medical therapy/smoking cessation and treatment strategy. The model was validated using a separate dataset of 254 patients. This model was then used to build a prototype interactive decision support system which was evaluated using calibration curves, decision curve analyses and area under the receiver operator characteristic (AUROC) curves. Results The AUROC curves demonstrated excellent discrimination for the risk of progression to chronic limb threatening ischaemia at two years(0.892) and five years(0.866) and likelihood of major adverse cardiovascular events(0.836), major adverse limb events(0.891) and revascularisation(0.896)within 5 years, regardless of the treatment strategy. Calibration curves demonstrated good consistency between predicted and actual outcomes and decision curve analyses confirmed clinical utility. The tool maintained an accuracy of >80% and an effect size of >0.5. Conclusion Our decision support system successfully predicts outcomes in IC regardless of treatment strategy, offering potential for improved risk stratification and patient outcomes.

Novel Artificial Intelligence-Based Technology to Diagnose Asthma Using Methacholine Challenge Tests.

The methacholine challenge test (MCT) has high sensitivity but relatively low specificity for asthma diagnosis. This study aimed to develop and validate machine learning (ML) models to improve the diagnostic performance of MCT for asthma. Data from 1,501 patients with asthma symptoms who underwent MCT between 2015 and 2020 were analyzed. The patients were grouped as either the training (80%, n = 1,265) and test sets (20%, n = 236) depending on the time of referral. The conventional model (provocative concentration that causes a 20% decrease in forced expiratory volume in one second [FEV1]; PC20 ≤ 16 mg/mL) was compared with the prediction models derived from five ML methods: logistic regression, support vector machine, random forest, extreme gradient boosting, and artificial neural network. The area under the receiver operator characteristic curves (AUROC) and area under the precision-recall curves (AUPRC) of each model were compared. The prediction models were further analyzed using different input combinations of FEV1, forced vital capacity (FVC), and forced expiratory flow at 25%-75% of forced vital capacity (FEF25%-75%) values obtained during MCT. In total, 545 patients (36.3%) were diagnosed with asthma. The AUROC of the conventional model was 0.856 (95% confidence interval [CI], 0.852-0.861), and the AUPRC was 0.759 (95% CI, 0.751-0.766). All the five ML prediction models had higher AUROC and AUPRC values than those of the conventional model, and random forest showed both highest AUROC (0.950; 95% CI, 0.948-0.952) and AUROC (0.909; 95% CI, 0.905-0.914) when FEV1, FVC, and FEF25%-75% were included as inputs. Artificial intelligence-based models showed excellent performance in asthma prediction compared to using PC20 ≤ 16 mg/mL. The novel technology could be used to enhance the clinical diagnosis of asthma.

Blood-based Transcriptomic and Proteomic Biomarkers of Emphysema.

Rationale: Emphysema is a chronic obstructive pulmonary disease phenotype with important prognostic implications. Identifying blood-based biomarkers of emphysema will facilitate early diagnosis and development of targeted therapies. Objectives: To discover blood omics biomarkers for chest computed tomography-quantified emphysema and develop predictive biomarker panels. Methods: Emphysema blood biomarker discovery was performed using differential gene expression, alternative splicing, and protein association analyses in a training sample of 2,370 COPDGene participants with available blood RNA sequencing, plasma proteomics, and clinical data. Internal validation was conducted in a COPDGene testing sample (n = 1,016), and external validation was done in the ECLIPSE study (n = 526). Because low body mass index (BMI) and emphysema often co-occur, we performed a mediation analysis to quantify the effect of BMI on gene and protein associations with emphysema. Elastic net models with bootstrapping were also developed in the training sample sequentially using clinical, blood cell proportions, RNA-sequencing, and proteomic biomarkers to predict quantitative emphysema. Model accuracy was assessed by the area under the receiver operating characteristic curves for subjects stratified into tertiles of emphysema severity. Measurements and Main Results: Totals of 3,829 genes, 942 isoforms, 260 exons, and 714 proteins were significantly associated with emphysema (false discovery rate, 5%) and yielded 11 biological pathways. Seventy-four percent of these genes and 62% of these proteins showed mediation by BMI. Our prediction models demonstrated reasonable predictive performance in both COPDGene and ECLIPSE. The highest-performing model used clinical, blood cell, and protein data (area under the receiver operating characteristic curve in COPDGene testing, 0.90; 95% confidence interval, 0.85-0.90). Conclusions: Blood transcriptome and proteome-wide analyses revealed key biological pathways of emphysema and enhanced the prediction of emphysema.

A Deep-Learning Algorithm to Predict Short-Term Progression to Geographic Atrophy on Spectral-Domain Optical Coherence Tomography

The identification of patients at risk of progressing from intermediate age-related macular degeneration (iAMD) to geographic atrophy (GA) is essential for clinical trials aimed at preventing disease progression. DeepGAze is a fully automated and accurate convolutional neural network-based deep learning algorithm for predicting progression from iAMD to GA within 1 year from spectral-domain optical coherence tomography (SD-OCT) scans. To develop a deep-learning algorithm based on volumetric SD-OCT scans to predict the progression from iAMD to GA during the year following the scan. This retrospective cohort study included participants with iAMD at baseline and who either progressed or did not progress to GA within the subsequent 13 months. Participants were included from centers in 4 US states. Data set 1 included patients from the Age-Related Eye Disease Study 2 AREDS2 (Ancillary Spectral-Domain Optical Coherence Tomography) A2A study (July 2008 to August 2015). Data sets 2 and 3 included patients with imaging taken in routine clinical care at a tertiary referral center and associated satellites between January 2013 and January 2023. The stored imaging data were retrieved for the purpose of this study from July 1, 2022, to February 1, 2023. Data were analyzed from May 2021 to July 2023. A position-aware convolutional neural network with proactive pseudointervention was trained and cross-validated on Bioptigen SD-OCT volumes (data set 1) and validated on 2 external data sets comprising Heidelberg Spectralis SD-OCT scans (data sets 2 and 3). Prediction of progression to GA within 13 months was evaluated with area under the receiver-operator characteristic curves (AUROC) as well as area under the precision-recall curve (AUPRC), sensitivity, specificity, positive predictive value, negative predictive value, and accuracy. The study included a total of 417 patients: 316 in data set 1 (mean [SD] age, 74 [8]; 185 [59%] female), 53 in data set 2, (mean [SD] age, 83 [8]; 32 [60%] female), and 48 in data set 3 (mean [SD] age, 81 [8]; 32 [67%] female). The AUROC for prediction of progression from iAMD to GA within 1 year was 0.94 (95% CI, 0.92-0.95; AUPRC, 0.90 [95% CI, 0.85-0.95]; sensitivity, 0.88 [95% CI, 0.84-0.92]; specificity, 0.90 [95% CI, 0.87-0.92]) for data set 1. The addition of expert-annotated SD-OCT features to the model resulted in no improvement compared to the fully autonomous model (AUROC, 0.95; 95% CI, 0.92-0.95; P = .19). On an independent validation data set (data set 2), the model predicted progression to GA with an AUROC of 0.94 (95% CI, 0.91-0.96; AUPRC, 0.92 [0.89-0.94]; sensitivity, 0.91 [95% CI, 0.74-0.98]; specificity, 0.80 [95% CI, 0.63-0.91]). At a high-specificity operating point, simulated clinical trial recruitment was enriched for patients progressing to GA within 1 year by 8.3- to 20.7-fold (data sets 2 and 3). The fully automated, position-aware deep-learning algorithm assessed in this study successfully predicted progression from iAMD to GA over a clinically meaningful time frame. The ability to predict imminent GA progression could facilitate clinical trials aimed at preventing the condition and could guide clinical decision-making regarding screening frequency or treatment initiation.

271-OR: Capillary Blood Ketone Level and Future Ketoacidosis Risk in Type 1 Diabetes Using Sodium–Glucose Linked Transporter Inhibitors

Adjunctive-to-insulin Sodium-Glucose Linked Transporter inhibitors (SGLTi) improve metabolic control in T1D but cause diabetic ketoacidosis (DKA). We aimed to determine if time-varying ketone levels could predict future DKA. We explored data on 1194 participants randomly assigned to empagliflozin in a T1D trial program (NCT02414958 and NCT02580591). Protocol called for fasted capillary blood ketone levels 2-3 times/week, categorized into maximum, mean, and last values for sequential 1-month ‘index test’ periods during 6-12-month follow-up. Trial-adjudicated DKA/Ketosis was determined in subsequent 1-month ‘outcome’ periods. Area under the Receiver Operator Characteristic curves (AROC) were calculated. Gradient-boosted trees determined if prediction could be improved. Participants had mean age 44.3 years, HbA1c 8.14±0.60 percent, and contributed 3.70±2.97 ketone measurements/week. 617 outcomes (568 ketosis, 49 DKA events) occurred over follow-up. On-treatment ketone levels were higher for index test periods preceding an outcome period with an event compared to those without (for example, maximum ketone was 1.32±1.03 vs 0.80±0.71 mmol/l, respectively, p<0.001). AROC for maximum, mean and last ketone levels were 0.764, 0.755, and 0.682 respectively. Maximum ketone level ≥ 0.9 mmol/l had sensitivity 59%, specificity 84%, likelihood ratio positive 3.65, likelihood ratio negative 0.49, and diagnostic odds ratio 7.39. Inclusion of the other ketone measures (mean or last) or clinical variables did not improve the predictive validity of maximum ketone levels on their own. Routine surveillance of morning capillary ketone levels in T1D patient using SGLTi can predict 1-month risk of DKA/Ketosis and could allow for interventions to mitigate risk. Disclosure P.Bapat: None. D.Cherney: Other Relationship; Boehringer Ingelheim-Lilly, Merck, AstraZeneca, Sanofi, Mitsubishi-Tanabe, Abbvie, Janssen, Bayer, Prometic, BMS, Maze, Gilead, CSL-Behring, Otsuka, Novartis, Youngene, Lexicon and Novo-Nordisk, Research Support; Boehringer Ingelheim-Lilly, Merck, Janssen, Sanofi, AstraZeneca, CSL-Behring and Novo-Nordisk. D.Mumford: None. G.Tomlinson: None. L.Lovblom: None. B.A.Perkins: Advisory Panel; Dexcom, Inc., Insulet Corporation, Novo Nordisk, Sanofi, Vertex Pharmaceuticals Incorporated, Other Relationship; Abbott, Medtronic, Sanofi, Research Support; Novo Nordisk, Bank of Montreal (BMO). S.Dhaliwal: None. C.Song: None. D.R.Budhram: None. A.M.K.Bakhsh: None. D.Scarr: None. A.Weisman: None. M.Fralick: None. N.Ivers: Consultant; Novo Nordisk Canada Inc., IQVIA Inc. Funding Diabetes Canada (OG-3-21-5572-BP)

Automated Detection of Posterior Vitreous Detachment on OCT Using Computer Vision and Deep Learning Algorithms.

To develop automated algorithms for the detection of posterior vitreous detachment (PVD) using OCT imaging. Evaluation of a diagnostic test or technology. Overall, 42 385 consecutive OCT images (865 volumetric OCT scans) obtained with Heidelberg Spectralis from 865 eyes from 464 patients at an academic retina clinic between October 2020 and December 2021 were retrospectively reviewed. We developed a customized computer vision algorithm based on image filtering and edge detection to detect the posterior vitreous cortex for the determination of PVD status. A second deep learning (DL) image classification model based on convolutional neural networks and ResNet-50 architecture was also trained to identify PVD status from OCT images. The training dataset consisted of 674 OCT volume scans (33 026 OCT images), while the validation testing set consisted of 73 OCT volume scans (3577 OCT images). Overall, 118 OCT volume scans (5782 OCT images) were used as a separate external testing dataset. Accuracy, sensitivity, specificity, F1-scores, and area under the receiver operator characteristic curves (AUROCs) were measured to assess the performance of the automated algorithms. Both the customized computer vision algorithm and DL model results were largely in agreement with the PVD status labeled by trained graders. The DL approach achieved an accuracy of 90.7% and an F1-score of 0.932 with a sensitivity of 100% and a specificity of 74.5% for PVD detection from an OCT volume scan. The AUROC was 89% at the image level and 96% at the volume level for the DL model. The customized computer vision algorithm attained an accuracy of 89.5% and an F1-score of 0.912 with a sensitivity of 91.9% and a specificity of 86.1% on the same task. Both the computer vision algorithm and the DL model applied on OCT imaging enabled reliable detection of PVD status, demonstrating the potential for OCT-based automated PVD status classification to assist with vitreoretinal surgical planning. Proprietary or commercial disclosure may be found after the references.

Diagnostic accuracy of enhanced liver fibrosis test for nonalcoholic steatohepatitis-related fibrosis: Multicenter study.

The enhanced liver fibrosis (ELF) test is a noninvasive method for diagnosing hepatic fibrosis in patients with nonalcoholic fatty liver disease (NAFLD). This multicenter cohort study aimed to evaluate the accuracy of the ELF test and compare it with other noninvasive tests in Japan. We analyzed 371 Japanese patients with biopsy-proven NAFLD. We constructed area under the receiver operator characteristic curves (AUROC) to determine the diagnostic accuracies of the ELF test, the Mac-2-binding protein glycosylation isomer (M2BPGi), the Fibrosis-4 (FIB-4) index, and combinations of these indices. In patients with F0/F1/F2/F3/F4 fibrosis, the median values of the ELF test were 8.98/9.56/10.39/10.92/11.41, respectively. The AUROCs of the ELF test for patients with F0 versus F1-4, F0-1 versus F2-4, F0-2 versus F3-4, and F0-3 versus F4 fibrosis were 0.825/0.817/0.802/0.812, respectively. The AUROCs of the ELF test were greater than those of the FIB-4 index and M2BPGi at each fibrosis stage. Respective low and high cut-off values yielded sensitivities and specificities for predicting advanced fibrosis (≥F3) of 91.1% and 50.8%, and 38.5% and 92.8%, respectively. For F3 or F4 fibrosis, the combined values from the ELF test and FIB-4 index showed a sensitivity of 98.5%, and the combined values from the ELF test and M2BPGi assay showed a specificity of 97.5%. In Japan, the ELF test predicts NAFLD-related fibrosis from its early stages. The diagnostic ability of the ELF test was not inferior to that of other indices, and the combined values of ELF plus other indices were more accurate.

Diaphragmatic ultrasonography-based rapid shallow breathing index for predicting weaning outcome during a pressure support ventilation spontaneous breathing trial

BackgroundThe optimum timing to wean is crucial to avoid negative outcomes for mechanically ventilated patients. The rapid shallow breathing index (RSBI), a widely used weaning index, has limitations in predicting weaning outcomes. By replacing the tidal volume of the RSBI with diaphragmatic excursion (DE) and diaphragm thickening fraction (DTF) assessed by ultrasonography, we calculated two weaning indices, the diaphragmatic excursion rapid shallow breathing index (DE-RSBI, respiratory rate [RR]/DE) and the diaphragm thickening fraction rapid shallow breathing index (DTF-RSBI, RR/DTF). The aim of this study was to evaluate the predictive values of DTF-RSBI, DE-RSBI and traditional RSBI in weaning failure.MethodsThis prospective observational study included patients undergoing mechanical ventilation (MV) for > 48 h and who were readied for weaning. During a pressure support ventilation (PSV) spontaneous breathing trial (SBT), right hemidiaphragmatic excursion and DTF were measured by bedside ultrasonography as well as RSBI. Weaning failure was defined as: (1) failing the SBT and (2) SBT success but inability to maintain spontaneous breathing for more than 48 h without noninvasive or invasive ventilation. A receiver operator characteristic (ROC) curve was used for analyzing the diagnostic accuracy of RSBI, DE-RSBI, and DTF-RSBI.ResultsOf the 110 patients studied, 37 patients (33.6%) failed weaning. The area under the ROC (AUROC) curves for RSBI, DE-RSBI, and DTF-RSBI for predicting failed weaning were 0.639, 0.813, and 0.859, respectively. The AUROC curves for DE-RSBI and DTF-RSBI were significantly higher than for RSBI (P = 0.004 and P < 0.001, respectively). The best cut-off values for predicting failed weaning were RSBI > 51.2 breaths/min/L, DE-RSBI > 1.38 breaths/min/mm, and DTF-RSBI > 78.1 breaths/min/%.ConclusionsIn this study, two weaning indices determined by bedside ultrasonography, the DE-RSBI (RR/DE) and DTF-RSBI (RR/DTF), were shown to be more accurate than the traditional RSBI (RR/VT) in predicting weaning outcome during a PSV SBT.

A preoperative prediction model based on Lymphocyte-C-reactive protein ratio predicts postoperative anastomotic leakage in patients with colorectal carcinoma: a retrospective study

Background & AimsLymphocyte-C-reactive Protein Ratio (LCR) has been demonstrated as a promising new marker for predicting surgical and oncological outcomes in colorectal carcinoma (CRC). However, anastomotic leakage (AL) is also likely related to this inflammatory marker. Herein, we aimed to identify preoperative predictors of AL and build and develop a novel model able to identify patients at risk of developing AL.MethodsWe collected 858 patients with CRC undergoing elective radical operation between 2007 and 2018 at a single center were retrospectively reviewed. We performed univariable and multivariable analyses and built a multivariable model that predicts AL based on preoperative factors. Propensity adjustment was used to correct the bias introduced by non-random matching of the LCR. The model's performance was evaluated by using the area under the receiver operator characteristic curves (AUROCs), decision curve analysis (DCA), Brier scores, D statistics, and R2 values.ResultsAge, nutrition risk screening 2002 (NRS2002) score, tumor location and LCR, together with hemoglobin < 90 g/l, were independent predictors of AL. The models built on these variables showed good performance (internal validation: c-statistic = 0.851 (95%CI 0.803–0.965), Brier score = 0.049; temporal validation: c-statistic = 0.777 (95%CI 0.823–0.979), Brier score = 0.096). A regression equation to predict the AL was also established by multiple linear regression analysis: [Age(≥ 60 year) × 1.281] + [NRS2002(≥ 3) × 1.341] + [Tumor location(pt.) × 1.348]-[LCR(≤ 6000) × 1.593]-[Hemoglobin(< 90 g/L) × 1.589]-6.12.ConclusionPreoperative LCR is an independent predictive factor for AL. A novel model combining LCR values, age, tumor location, and NRS2002 provided an excellent preoperative prediction of AL in patients with CRC. The nomogram can help clinical decision-making and support future research.

FullROC: An R package for generating and analyzing eyewitness-lineup ROC curves.

A police lineup is a procedure in which a suspect is surrounded by known-innocent persons (fillers) and presented to the witness for an identification attempt. The purpose of a lineup is to test the investigator's hypothesis that the suspect is the culprit, and the investigator uses the witness' identification decision and the associated confidence level to inform this hypothesis. Whereas suspect identifications provide evidence of guilt, filler identifications and rejections provide evidence of innocence. Despite the capacity of lineups to provide exculpatory information, past research has focused, almost exclusively, on inculpatory behaviors. We recently developed a method for incorporating all lineup outcomes in a single receiver operator characteristic (ROC) curve. The area under the full lineup ROC curve reflects the total capacity of a lineup procedure to discriminate guilty suspects from innocent suspects. Here, we introduce a Comprehensive R Archive Network (CRAN) package, fullROC, to support eyewitness researchers in using the full ROC approach to analyze lineup data. The fullROC package provides functions for adjusting identification rates, generating full ROC curves for lineup data, computing the area under the ROC curves (AUC), and statistically comparing the AUCs of different lineups. Using both simulated and empirical data, we illustrate the functionality of the fullROC CRAN package. In brief, the fullROC package provides a useful tool for eyewitness researchers to analyze lineup data using the full ROC method, which incorporates both the inculpatory and exculpatory information of eyewitness behaviors.

Diagnostic performance of dual-energy CT in nonspecific terminal ileitis.

To investigate the value of dual-energy CT (DECT) imaging in the diagnosis of nonspecific terminal ileitis (NTI). This is a retrospective study, enrolling patients with symptomatic terminal ileitis that underwent conventional multidetector CT (MDCT) enterography or DECT enterography. The sensitivity of the diagnosis of NTI between MDCT images and different kinds of DECT images (40-70 kev virtual monoenergetic images (VMIs) and iodine density images) was compared. The iodine concentrations of lesion bowel wall among NTI, Crohn's disease (CD) and intestinal tuberculosis (ITB) in DECT group and the value of lesion-to-non-lesion contrast ratios of all patients were measured. Receiver operator characteristic (ROC) curves for normalized iodine concentration (NIC) for differentiating among the three kinds of disease were drawn. The sensitivity for the diagnosis of NTI in DECT group (including 40 kev, 50keV VMIs, and iodine density images) were all 89.7%, significantly higher than that in MDCT group (65.1%) (P = 0.026). Statistical analysis did not reveal marked differences between 60 kev, 70 kev VMIs (86.2%) and MDCT images (65.1%) (P = 0.059). The NIC of NTI was (0.15 ± 0.04)100µg/cm3 and (0.45 ± 0.08)100µg/cm3, significantly lower than that of CD (0.34 ± 0.09) 100µg/cm3, (0.85 ± 0.06) 100µg/cm3 and that of ITB (0.29 ± 0.07) 100µg/cm3, (0.88 ± 0.07) 100µg/cm3 at the enteric phase (EP) and portal venous phase (PVP) (P < 0.001, wholly). The area under the ROC curves (AUROCs) of NICEP and NICPVP were 0.910 and 0.980, respectively, for differentiating between NTI and CD. The value of lesion-to-non-lesion contrast ratios is maximum on the 40keV VMI both EP and PVP. The value of lesion-to-non-lesion contrast ratios of NTI was lower than that of CD and ITB on each image. The AUROCs of NICEP and NICPVP were 0.875 and 0.940, respectively, for differentiating between NTI and ITB. DECT has higher sensitivity in the diagnosis of NTI than MDCT. Low-keV VMI and iodine density images of DECT can clearly show the NTI. DECT imaging can help to differentiate NTI from CD or ITB by comparing the NIC.

Acute-phase serum amyloid A for early detection of hepatocellular carcinoma in cirrhotic patients with low AFP level

Regular hepatocellular carcinoma (HCC) surveillance by ultrasonography in combination with the α-fetoprotein (AFP) examination is unsatisfactory in diagnostic sensitivity for early-stage HCC especially in cirrhotic patients. We conducted a prospective study in a tertiary medical center in Taiwan and consecutively collected serum samples from patients with chronic hepatitis, liver cirrhosis (LC), or HCC for new biomarker discovery. Overall, 166 patients were enrolled, including 40 hepatitis, 30 LC, and 96 HCC. Four acute-phase serum amyloid A (A-SAA) derived biomarkers including total A-SAA, A-SAA monomer and oligomer, and protein misfolding cyclic amplification (PMCA) signal were measured and compared between patients with and without HCC. A-SAA biomarkers significantly increased in the HCC group when compared to the hepatitis and LC groups, and generally increased in more advanced tumor stages. Among A-SAA biomarkers, the area under the receiver operator characteristic curves (AUROCs) for PMCA signal in discrimination of all-stage and early-stage HCC were 0.86 and 0.9 in cirrhotic patients, which is comparable to AFP. For cirrhotic patients with low AFP (< 7 ng/mL), PMCA signal maintained good capacity in prediction of early-stage HCC (AUROC: 0.94). Serum A-SAA and its prion-like property showed a potential to complement AFP in detection of early-stage HCC.

Using noninvasive anthropometric indices to develop and validate a predictive model for metabolic syndrome in Chinese adults: a nationwide study.

PurposeMetabolic syndrome (Mets) is a pathological condition that includes many abnormal metabolic components and requires a simple detection method for rapid use in a large population. The aim of the study was to develop a diagnostic model for Mets in a Chinese population with noninvasive anthropometric and demographic predictors.Patients and methodsLeast absolute shrinkage and selection operator (LASSO) regression was used to screen predictors. A large sample from the China National Diabetes and Metabolic Disorders Survey (CNDMDS) was used to develop the model with logistic regression, and internal, internal-external and external validation were conducted to evaluate the model performance. A score calculator was developed to display the final model.ResultsWe evaluated the discrimination and calibration of the model by receiver operator characteristic (ROC) curves and calibration curve analysis. The area under the ROC curves (AUCs) and the Brier score of the original model were 0.88 and 0.122, respectively. The mean AUCs and the mean Brier score of 10-fold cross validation were 0.879 and 0.122, respectively. The mean AUCs and the mean Brier score of internal–external validation were 0.878 and 0.121, respectively. The AUCs and Brier score of external validation were 0.862 and 0.133, respectively.ConclusionsThe model developed in this study has good discrimination and calibration performance. Its stability was proved by internal validation, external validation and internal-external validation. Then, this model has been displayed by a calculator which can exhibit the specific predictive probability for easy use in Chinese population.

Predictive model based on gene and laboratory data for intravenous immunoglobulin resistance in Kawasaki disease in a Chinese population

BackgroundHere, we investigated the predictive efficiency of a newly developed model based on single nucleotide polymorphisms (SNPs) and laboratory data for intravenous immunoglobulin (IVIG) resistance in Kawasaki disease (KD) in a Chinese population.MethodsData relating to children with KD were acquired from a single center between December 2015 and August 2019 and used to screen target SNPs. We then developed a predictive model of IVIG resistance using previous laboratory parameters. We then validated our model using data acquired from children with KD attending a second center between January and December 2019.ResultsAnalysis showed that rs10056474 GG, rs746994GG, rs76863441GT, rs16944 (CT/TT), and rs1143627 (CT/CC), increased the risk of IVIG-resistance in KD patients (odds ratio, OR > 1). The new predictive model, which combined SNP data with a previous model derived from laboratory data, significantly increased the area under the receiver-operator-characteristic curves (AUC) (0.832, 95% CI: 0.776-0.878 vs 0.793, 95%CI:0.734-0.844, P < 0.05) in the development dataset, and (0.820, 95% CI: 0.730-0.889 vs 0.749, 95% CI: 0.652-0.830, P < 0.05) in the validation dataset. The sensitivity and specificity of the new assay were 65.33% (95% CI: 53.5-76.0%) and 86.67% (95% CI: 80.2-91.7%) in the development dataset and 77.14% (95% CI: 59.9-89.6%) and 86.15% (95% CI: 75.3-93.5%) in the validation dataset.ConclusionAnalysis showed that rs10056474 and rs746994 in the SMAD5 gene, rs76863441 in the PLA2G7 gene, and rs16944 or rs1143627 in the interleukin (IL)-1B gene, were associated with IVIG resistant KD in a Chinese population. The new model combined SNPs with laboratory data and improved the predictve efficiency of IVIG-resistant KD.

1085-P: Defining the Role of Insulin Clearance in Dysglycemia: The Microbiome and Insulin Longitudinal Evaluation Study (MILES)

Insulin resistance and insufficient insulin secretion lead to type 2 diabetes. A role of reduced insulin clearance has been suggested, but few studies have investigated the contribution of insulin clearance while simultaneously examining insulin sensitivity and secretion. The goal of this study was to conduct such an investigation in 224 non-Hispanic White and 129 African Americans individuals from MILES (mean age 59, 62% female). Participants underwent oral glucose tolerance tests from which insulin sensitivity, insulin secretion, insulin clearance, and disposition index were calculated. Logistic regression models (controlling for age, sex, race, and BMI) examined the individual contributions of these traits separately, as well as the joint contribution of insulin sensitivity, clearance, and secretion, to early dysglycemia (prediabetes plus newly diagnosed diabetes; 46% of cohort). Models used standardized predictors such that odds ratios (OR) represent the change in odds of dysglycemia per standard deviation change. Model performance was assessed via Akaike’s Information Criterion (AIC) and area under the receiver operator characteristic curves (AUROC). In separate models, dysglycemia was associated with insulin sensitivity (OR 0.2; 0.14-0.30; P&amp;lt;.001), disposition index (OR 0.13; 0.08-0.20; P&amp;lt;.001) and insulin clearance (OR 0.66; 0.51-0.87; P&amp;lt;.001). In a joint model, only reduced insulin sensitivity (OR 0.04; 0.02-0.08; P&amp;lt;.001) and secretion (OR 0.15; 0.08-0.27; P&amp;lt;.001) were associated with dysglycemia. Models with insulin sensitivity (AIC=376.7; AUROC=0.82), disposition index (AIC=324.8; AUROC=0.87), or three insulin traits (AIC=303.4; AUROC=0.89) had the highest discriminative value. These results suggest that in the race groups studied, insulin resistance and compromised insulin secretion are the main independent defects leading to dysglycemia, while reduced insulin clearance may have a minor contributory role. Disclosure A. Wood: None. E. T. Jensen: None. G. Ramesh: None. Z. Arzumanyan: None. K. Lam: None. A. Bertoni: None. J. I. Rotter: None. Y. Chen: None. M. O. Goodarzi: None. Funding National Institutes of Health (R01DK109588)

Classification of Comprehensive Neuro-Ophthalmologic Measures of Postacute Concussion

Symptom-based methods of concussion diagnosis in contact sports result in underdiagnosis and repeated head injury exposure, increasing the risk of long-term disability. Measures of neuro-ophthalmologic (NO) function have the potential to serve as objective aids, but their diagnostic utility is unknown. To identify NO measures that accurately differentiate athletes with and without concussion. This cohort study was conducted among athletes with and without concussion who were aged 17 to 22 years between 2016 and 2017. Eye movements and cognitive function were measured a median of 19 days after injury among patients who had an injury meeting the study definition of concussion while playing a sport (retrospectively selected from a concussion clinic), then compared with a control group of participants without concussion (enrolled from 104 noncontact collegiate athlete volunteers without prior head injury). Data analysis was conducted from November 2019 through May 2020. Concussion. Classification accuracy of clinically important discriminator eye-tracking (ET) metrics. Participants' eye movements were evaluated with a 12-minute ET procedure, yielding 42 metrics related to smooth pursuit eye movement (SPEM), saccades, dynamic visual acuity, and reaction time. Clinically important discriminator metrics were defined as those with significantly different group differences and area under the receiver operator characteristic curves (AUROCs) of at least 0.70. A total of 34 participants with concussions (mean [SD] age, 19.7 [2.4] years; 20 [63%] men) and 54 participants without concussions (mean [SD] age, 20.8 [2.2] years; 31 [57%] men) completed the study. Six ET metrics (ie, simple reaction time, discriminate reaction time, discriminate visual reaction speed, choice visual reaction speed, and reaction time on 2 measures of dynamic visual acuity 2) were found to be clinically important; all were measures of reaction time, and none were related to SPEM. Combined, these 6 metrics had an AUROC of 0.90 (95% CI, 0.80-0.99), a sensitivity of 77.8%, and a specificity of 92.6%. The 6 metrics remained significant on sensitivity testing. In this study, ET measures of slowed visual reaction time had high classification accuracy for concussion. Accurate, objective measures of NO function have the potential to improve concussion recognition and reduce the disability associated with underdiagnosis.

A Machine Learning-Based Model to Predict Survival After Transarterial Chemoembolization for BCLC Stage B Hepatocellular Carcinoma.

ObjectiveWe sought to develop and validate a novel prognostic model for predicting survival of patients with Barcelona Clinic Liver Cancer Stages (BCLC) stage B hepatocellular carcinoma (HCC) using a machine learning approach based on random survival forests (RSF).MethodsWe retrospectively analyzed overall survival rates of patients with BCLC stage B HCC using a training (n = 602), internal validation (n = 301), and external validation (n = 343) groups. We extracted twenty-one clinical and biochemical parameters with established strategies for preprocessing, then adopted the RSF classifier for variable selection and model development. We evaluated model performance using the concordance index (c-index) and area under the receiver operator characteristic curves (AUROC).ResultsRSF revealed that five parameters, namely size of the tumor, BCLC-B sub-classification, AFP level, ALB level, and number of lesions, were strong predictors of survival. These were thereafter used for model development. The established model had a c-index of 0.69, whereas AUROC for predicting survival outcomes of the first three years reached 0.72, 0.71, and 0.73, respectively. Additionally, the model had better performance relative to other eight Cox proportional-hazards models, and excellent performance in the subgroup of BCLC-B sub-classification B I and B II stages.ConclusionThe RSF-based model, established herein, can effectively predict survival of patients with BCLC stage B HCC, with better performance than previous Cox proportional hazards models.

Modelling the Validity of Periodontal Disease Screening Questions in a Nondental Clinical Setting

ObjectivePeriodontal examinations are time-consuming and potentially uncomfortable for recipients. We modelled if self-reported questions alone, or combined with objective evidence of periodontal bone loss observable from radiographs, are accurate predictors of periodontitis. MethodsSelf-reported data from the Australian National Survey of Adult Oral Heath 2004-06 were compared with clinical periodontal examinations to assess the validity of 8 periodontitis screening questions in predicting moderate/severe periodontitis. To model alveolar bone loss, a proxy variable simulating radiographic clinical attachment level (rCAL) was created. Three multivariable binary logistic regression models were constructed: responses to 8 screening questions alone (Model 1), screening questions combined with 5 classic periodontitis risk indicators (age, sex, smoking status, country of birth, and diabetes status) (Model 2), and the addition of rCAL (Model 3). Predictive validity was determined via sensitivity (Se) and specificity (Sp) scores and graphically represented using area under the receiver operator characteristic curves (AUROC). ResultsData from 3630 participants periodontally examined determined that 32.4% exhibited periodontitis. Periodontitis risk indicators were all significantly associated with periodontitis case status. Six of 8 screening questions (Model 1) were weak periodontitis predictors (Se = 0.28; Sp = 0.89; AUROC = 0.61). Combining 13 variables for (Model 2) improved prediction (Se = 0.55; Sp = 0.81; AUROC = 0.77). The addition of rCAL (Model 3) improved diagnostic capacity considerably (AUROC = 0.86). ConclusionsSelf-reported questions combined with classic risk indicators are “useful” for periodontitis screening. Addition of radiographs markedly improved diagnostic validity. Based on modelling, nondental health care professionals may provisionally screen for periodontitis with minimal training.