Diagnostic Performance Measures Research Articles

The Whole is Greater Than the Sum of its Parts: GCS Versus GCS-Motor for Triage in Geriatric Trauma

BackgroundTrauma field triage matches injured patients to the appropriate level of care. Prior work suggests the Glasgow Coma Scale motor (GCSm) is as accurate as the total GCS (GCSt) and easier to use. However, older patients present with higher GCS for a given injury, and as such, it is unclear if this substitution is advisable. Our objective was to compare the GCS deficit patterns between geriatric and adult patients presenting with severe traumatic brain injury (TBI), as well as the diagnostic performance of the GCSm versus GCSt within the field triage criteria in these populations. Materials and MethodsWe conducted a retrospective, observational cohort study of patients ≥16 y in the National Trauma Data Bank 2007–2015. GCS deficit patterns were compared between adults (16-65) and geriatric patients (>65). Measures of diagnostic performance of GCSt≤13 versus GCSm≤5 criteria to predict trauma center need (TCN) were compared. ResultsIn total, 4,480,185 patients were analyzed (28% geriatric). Geriatric patients more frequently presented with non–motor–only deficits than adults (16.4% versus 12.4%, P < 0.001), and these patients demonstrated higher severe TBI (40.3% versus 36.7%, P < 0.001) and craniotomy (5.8% versus 5.1%, P < 0.001) rates. GCSt was more sensitive and accurate in predicting TCN for geriatric patients and had lower rates of undertriage as compared to GCSm. ConclusionsGeriatric patients more frequently present with non–motor–only deficits after injury, and this is associated with severe head injury. Substitution of GCSm for GCSt would exacerbate undertriage in geriatric patients and, thus, the total GCS should be maintained for field triage in geriatric patients.

Postoperative Copeptin as a Biomarker for Development of Diabetes Insipidus Following Hypothalamic-Pituitary Surgery

ObjectiveCopeptin is a surrogate marker of arginine vasopressin release with better stability and simplicity of measurement. Postoperative copeptin levels may guide clinicians in stratifying patients who need close monitoring of fluid balance. The objective is to determine whether copeptin is a predictive marker of postoperative diabetes insipidus (DI). MethodsThis is a prospective diagnostic study. Patients who underwent neurosurgical intervention of the sellar-suprasellar regions were recruited. Serum copeptin levels were measured before and after surgery, within 24 hours. Logistic regression analysis and diagnostic performance measures were calculated to determine the relationship between postoperative copeptin levels and DI. ResultsOf 82 patients, 26 (31.7%) developed postoperative DI, with 7 patients (8.5%) having permanent DI. The samples for copeptin measurement were taken at 13 ± 2.1 hours postoperatively. From the receiver operating characteristic analysis, low postoperative copeptin levels (<2.5 pmol/L) demonstrated an acceptable ability to predict DI (area under the curve, 0.72; 95% CI, 0.60-0.84). Discriminative power was stronger in the permanent DI group (area under the curve, 0.82; 95% CI, 0.64-1.00). Postoperative copeptin levels <2.5 pmol/L were associated with DI (specificity > 91%). However, postoperative copeptin levels >20 pmol/L were rarely associated with DI, with a negative predictive value of 100%. ConclusionsIn patients undergoing sellar-suprasellar interventions, low postoperative copeptin levels within the first postoperative day predict postoperative DI, whereas high levels exclude it. Copeptin measurement should be applied in the clinical practice of postoperative care in patients following hypothalamic-pituitary surgery. This study may expand the potential use of copeptin, including in the Asian population.

Identifying Novel Biomarkers Ready for Evaluation in Low-Prevalence Populations for the Early Detection of Upper Gastrointestinal Cancers: A Systematic Review.

IntroductionDetecting upper gastrointestinal (GI) cancers in primary care is challenging, as cancer symptoms are common, often non-specific, and most patients presenting with these symptoms will not have cancer. Substantial investment has been made to develop biomarkers for cancer detection, but few have reached routine clinical practice. We aimed to identify novel biomarkers for upper GI cancers which have been sufficiently validated to be ready for evaluation in low-prevalence populations.MethodsWe systematically searched MEDLINE, Embase, Emcare, and Web of Science for studies published in English from January 2000 to October 2019 (PROSPERO registration CRD42020165005). Reference lists of included studies were assessed. Studies had to report on second measures of diagnostic performance (beyond discovery phase) for biomarkers (single or in panels) used to detect pancreatic, oesophageal, gastric, and biliary tract cancers. We included all designs and excluded studies with less than 50 cases/controls. Data were extracted on types of biomarkers, populations and outcomes. Heterogeneity prevented pooling of outcomes.ResultsWe identified 149 eligible studies, involving 22,264 cancer cases and 49,474 controls. A total of 431 biomarkers were identified (183 microRNAs and other RNAs, 79 autoantibodies and other immunological markers, 119 other proteins, 36 metabolic markers, 6 circulating tumour DNA and 8 other). Over half (n = 231) were reported in pancreatic cancer studies. Only 35 biomarkers had been investigated in at least two studies, with reported outcomes for that individual marker for the same tumour type. Apolipoproteins (apoAII-AT and apoAII-ATQ), and pepsinogens (PGI and PGII) were the most promising biomarkers for pancreatic and gastric cancer, respectively.ConclusionMost novel biomarkers for the early detection of upper GI cancers are still at an early stage of matureness. Further evidence is needed on biomarker performance in low-prevalence populations, in addition to implementation and health economic studies, before extensive adoption into clinical practice can be recommended.Electronic Supplementary MaterialThe online version of this article (10.1007/s12325-020-01571-z) contains supplementary material, which is available to authorized users.

Evaluation of binary diagnostic tests accuracy for medical researches

Abstract Objectives The aim of this study is to introduce the features of diagnostic tests. In addition, it will be demonstrated which performance measures can be used for diagnostic tests with binary results, the properties of these measures and how to interpret them. Materials and Methods The evaluation of the diagnostic test performance measures may differ depending on whether the test result is numerical or binary. When the diagnostic test result is continuous numerical data, ROC analysis is often utilized. The performance of a diagnostic test with binary results are usually evaluated using the measures of sensitivity and specificity. However, there are some important measures other than these two measures for binary test results. These measures are predictive values, overall accuracy, diagnostic odds ratio, Youden index, and likelihood ratios. Results A hypothetical data has been produced based on the studies conducted on the performance of rapid tests (Specific IgM/IgG) according to the RT-PCR test for Covid 19 in the literature. An example of a diagnostic test (Specific IgM/IgG) with a binary result is given and all measurements and their confidence interval are obtained for this data. The performance of rapid test was examined and interpreted. Conclusion It is important to design and evaluate the performance of diagnostic/screening tests for health care. In this review, some basic definitions, performance measures that can be used only in evaluating the diagnostic tests with binary results and their confidence intervals are mentioned. Having many different measures provides different interpretations in the evaluation of test performance. Accurately predicting the performance of a diagnostic test depends on many factors. These factors can be study design, criteria of participant selection, sample size calculation, test methods etc. There are guidelines that ensure that all information regarding the conditions under which the study was conducted is in report, in terms of such factors. Therefore, these guidelines are recommended for use of the checklist by many publishers.

Toluidine blue versus frozen section for assessment of mucosal tumor margins in oral squamous cell carcinoma

BackgroundWhen the resected specimen is sent for intraoperative margin assessment, all margins are grossly checked, and selected margins undergo a frozen section (FS) examination. Therefore, there is a possibility of sampling error. This study evaluated the effectiveness of using toluidine blue (TB) as an intraoperative triage screening tool to detect positive mucosal margins of the resected specimens of oral squamous cell carcinoma (OSCC) and serve as a guide for FS sampling.MethodsSurgical samples of 30 consecutive patients with biopsy-proven OSCC were included in the study. A total of 140 mucosal margins were analyzed intraoperatively by TB and FS, the results were compared with the final histopathology.ResultsOf the 140 examined mucosal tumor margins, 14 stained positives with TB, six were true-positives, eight were false-positives, and there were no false-negatives, as confirmed by final histopathology of the same margins. The diagnostic performance measures were sensitivity 100.0%; specificity 94.0%; positive predictive value (PPV) 42.9%; negative predictive value (NPV) 100.0%; and accuracy 94.3% (95% CI: 89.0–97.5%). For FS, there were three true-positives, three false-negatives, and no false-positives. The diagnostic performance measures were sensitivity 50.0%; specificity 100.0%; PPV 100.0%; NPV 97.8%; and accuracy 97.9% (95% CI: 93.9–99.6%).ConclusionTB is less specific but more sensitive than FS for detecting positive mucosal margins of resected OSCC. Screening the tumor mucosal margins with TB before FS sampling may help identify more tumor-bearing margins.Trial registrationThis trial was registered at ClinicalTrials.gov. Registration number: NCT03554967. Registration date: June 13, 2018.

Statistical Evaluation of Medical Tests

In this review, we present an overview of the main aspects related to the statistical evaluation of medical tests for diagnosis and prognosis. Measures of diagnostic performance for binary tests, such as sensitivity, specificity, and predictive values, are introduced, and extensions to the case of continuous-outcome tests are detailed. Special focus is placed on the receiver operating characteristic (ROC) curve and its estimation, with emphasis on the topic of covariate adjustment. The extension to the case of time-dependent ROC curves for evaluating prognostic accuracy is also touched upon. We apply several of the approaches described to a data set derived from a study aimed to evaluate the ability of homeostasis model assessment of insulin resistance (HOMA-IR) levels to identify individuals at high cardio-metabolic risk and how such discriminatory ability might be influenced by age and gender. We also outline software available for the implementation of the methods.

Symptom-Based Dispatching in an Emergency Medical Communication Centre: Sensitivity, Specificity, and the Area under the ROC Curve.

Background: Measuring the performance of emergency medical dispatch tools used in paramedic-staffed emergency medical communication centres (EMCCs) is rarely performed. The objectives of our study were, therefore, to measure the performance and accuracy of Geneva’s dispatch system based on symptom assessment, in particular, the performance of ambulance dispatching with lights and sirens (L&S) and to measure the effect of adding specific protocols for each symptom. Methods: We performed a prospective observational study including all emergency calls received at Geneva’s EMCC (Switzerland) from 1 January 2014 to 1 July 2019. The risk levels selected during the emergency calls were compared to a reference standard, based on the National Advisory Committee for Aeronautics (NACA) scale, dichotomized to severe patient condition (NACA ≥ 4) or stable patient condition (NACA < 4) in the field. The symptom-based dispatch performance was assessed using a receiver operating characteristic (ROC) curve. Contingency tables and a Fagan nomogram were used to measure the performance of the dispatch with or without L&S. Measurements were carried out by symptom, and a group of symptoms with specific protocols was compared to a group without specific protocols. Results: We found an acceptable area under the ROC curve of 0.7474, 95%CI (0.7448–0.7503) for the 148,979 assessments included in the study. Where the severity prevalence was 21%, 95%CI (20.8–21.2). The sensitivity of the L&S dispatch was 87.5%, 95%CI (87.1–87.8); and the specificity was 47.3%, 95%CI (47.0–47.6). When symptom-specific assessment protocols were used, the accuracy of the assessments was slightly improved. Conclusions: Performance measurement of Geneva’s symptom-based dispatch system using standard diagnostic test performance measurement tools was possible. The performance was found to be comparable to other emergency medical dispatch systems using the same reference standard. However, the implementation of specific assessment protocols for each symptom may improve the accuracy of symptom-based dispatch systems.

Natriuretic peptides for the detection of diastolic dysfunction and heart failure with preserved ejection fraction\u2014a systematic review and meta-analysis

BackgroundAn overview of the diagnostic performance of natriuretic peptides (NPs) for the detection of diastolic dysfunction (DD) and heart failure with preserved ejection fraction (HFpEF), in a non-acute setting, is currently lacking.MethodsWe performed a systematic literature search in PubMed and Embase.com (May 13, 2019). Studies were included when they (1) reported diagnostic performance measures, (2) are for the detection of DD or HFpEF in a non-acute setting, (3) are compared with a control group without DD or HFpEF or with patients with heart failure with reduced ejection fraction, (4) are in a cross-sectional design. Two investigators independently assessed risk of bias of the included studies according to the QUADAS-2 checklist. Results were meta-analysed when three or more studies reported a similar diagnostic measure.ResultsFrom 11,728 titles/abstracts, we included 51 studies. The meta-analysis indicated a reasonable diagnostic performance for both NPs for the detection of DD and HFpEF based on AUC values of approximately 0.80 (0.73–0.87; I2 = 86%). For both NPs, sensitivity was lower than specificity for the detection of DD and HFpEF: approximately 65% (51–85%; I2 = 95%) versus 80% (70–90%; I2 = 97%), respectively. Both NPs have adequate ability to rule out DD: negative predictive value of approximately 85% (78–93%; I2 = 95%). The ability of both NPs to prove DD is lower: positive predictive value of approximately 60% (30–90%; I2 = 99%).ConclusionThe diagnostic performance of NPs for the detection of DD and HFpEF is reasonable. However, they may be used to rule out DD or HFpEF, and not for the diagnosis of DD or HFpEF.

Variability in haemoglobin concentration by measurement tool and blood source: an analysis from seven countries

ObjectiveWe explore factors such as the blood sampling site (capillary vs venous), the equipment (HemoCue vs automated haematology analyser) and the model of the HemoCue device (201+ vs 301) that...

Use of Steroid Profiling Combined With Machine Learning for Identification and Subtype Classification in Primary Aldosteronism

Most patients with primary aldosteronism, a major cause of secondary hypertension, are not identified or appropriately treated because of difficulties in diagnosis and subtype classification. Applications of artificial intelligence combined with mass spectrometry-based steroid profiling could address this problem. To assess whether plasma steroid profiling combined with machine learning might facilitate diagnosis and treatment stratification of primary aldosteronism, particularly for patients with unilateral adenomas due to pathogenic KCNJ5 sequence variants. This diagnostic study was conducted at multiple tertiary care referral centers. Steroid profiles were measured from June 2013 to March 2017 in 462 patients tested for primary aldosteronism and 201 patients with hypertension. Data analyses were performed from September 2018 to August 2019. The aldosterone to renin ratio and saline infusion tests were used to diagnose primary aldosteronism. Subtyping was done by adrenal venous sampling and follow-up of patients who underwent adrenalectomy. Statistical tests and machine-learning algorithms were applied to plasma steroid profiles. Areas under receiver operating characteristic curves, sensitivity, specificity, and other diagnostic performance measures were calculated. Primary aldosteronism was confirmed in 273 patients (165 men [60%]; mean [SD] age, 51 [10] years), including 134 with bilateral disease and 139 with unilateral adenomas (58 with and 81 without somatic KCNJ5 sequence variants). Plasma steroid profiles varied according to disease subtype and were particularly distinctive in patients with adenomas due to KCNJ5 variants, who showed better rates of biochemical cure after adrenalectomy than other patients. Among patients tested for primary aldosteronism, a selection of 8 steroids in combination with the aldosterone to renin ratio showed improved effectiveness for diagnosis over either strategy alone. In contrast, the steroid profile alone showed superior performance over the aldosterone to renin ratio for identifying unilateral disease, particularly adenomas due to KCNJ5 variants. Among 632 patients included in the analysis, machine learning-designed combinatorial marker profiles of 7 steroids alone both predicted primary aldosteronism in 1 step and subtyped patients with unilateral adenomas due to KCNJ5 variants at diagnostic sensitivities of 69% (95% CI, 68%-71%) and 85% (95% CI, 81%-88%), respectively, and at specificities of 94% (95% CI, 93%-94%) and 97% (95% CI, 97%-98%), respectively. The validation series yielded comparable diagnostic performance. Machine learning-designed combinatorial plasma steroid profiles may facilitate both screening for primary aldosteronism and identification of patients with unilateral adenomas due to pathogenic KCNJ5 variants, who are most likely to show benefit from surgical intervention.

Pediatric appendiceal ultrasound: maintaining accuracy, increasing determinacy and improving clinical outcomes following the introduction of a standardized reporting template

Pediatric patients who underwent appendiceal US and received an equivocal interpretation had poorer clinical outcomes and higher medical costs compared to those to whom a definitive interpretation was given, either positive or negative. In an effort to reduce equivocal interpretations, we educated our group on the importance of increasing determinacy and encouraged the use of a reporting template with a definitive impression. We hypothesized that educational sessions and implementation of an optional reporting template with only a definitive impression would reduce equivocal reporting and improve clinical outcomes without negatively impacting US diagnostic performance. We retrospectively reviewed the charts of all patients <18years old at Mayo Clinic Rochester whose initial evaluation for acute appendicitis was a US in the 3-year period following educational sessions and template implementation. All studies were interpreted by board-certified fellowship-trained pediatric radiologists. We performed statistical analysis to compare the pre- and post-implementation cohorts. Following intervention, the rate of equivocal US interpretations was reduced from 23.7% to 9.3% (P<0.001). For studies with a definitive interpretation, measures of diagnostic performance of appendiceal US were similar for the pre- and post-implementation groups. US performance parameters were independent of appendiceal visualization. Follow-up CT utilization decreased from 18.7% to 8.9% (P<0.001). The negative laparotomy rate resulting from false-positive US interpretations remained low (6.8% vs. 5.0%, P=0.31). Following education sessions and implementation of an appendiceal US reporting template encouraging definitive reporting, equivocation was reduced, excellent diagnostic performance was maintained, follow-up CT utilization was reduced, and a low negative laparotomy rate was preserved.

Risk of bias in studies investigating novel diagnostic biomarkers for heart failure with preserved ejection fraction. A systematic review.

AimDiagnosing heart failure with preserved ejection fraction (HFpEF) in the non‐acute setting remains challenging. Natriuretic peptides have limited value for this purpose, and a multitude of studies investigating novel diagnostic circulating biomarkers have not resulted in their implementation. This review aims to provide an overview of studies investigating novel circulating biomarkers for the diagnosis of HFpEF and determine their risk of bias (ROB).Methods and resultsA systematic literature search for studies investigating novel diagnostic HFpEF circulating biomarkers in humans was performed up until 21 April 2020. Those without diagnostic performance measures reported, or performed in an acute heart failure population were excluded, leading to a total of 28 studies. For each study, four reviewers determined the ROB within the QUADAS‐2 domains: patient selection, index test, reference standard, and flow and timing. At least one domain with a high ROB was present in all studies. Use of case‐control/two‐gated designs, exclusion of difficult‐to‐diagnose patients, absence of a pre‐specified cut‐off value for the index test without the performance of external validation, the use of inappropriate reference standards and unclear timing of the index test and/or reference standard were the main bias determinants. Due to the high ROB and different patient populations, no meta‐analysis was performed.ConclusionThe majority of current diagnostic HFpEF biomarker studies have a high ROB, reducing the reproducibility and the potential for clinical care. Methodological well‐designed studies with a uniform reference diagnosis are urgently needed to determine the incremental value of circulating biomarkers for the diagnosis of HFpEF.

Prevalence of Metabolic Syndrome by different definitions, and its association with type 2 diabetes, pre-diabetes, and cardiovascular disease risk in Brazil

Background and aimsMetabolic Syndrome (MS) is increasing in developing countries. Different definitions of MS lead to discrepancies in prevalence estimates and applicability. We assessed the prevalence of MS as defined by the International Diabetes Federation (IDF), modified National Cholesterol Education Program Adult Treatment Plan III (Modified NCEP) and Joint Interim Statement (JIS); compared the diagnostic performance and association of these definitions of MS with pre-diabetes, type 2 diabetes mellitus (T2DM) and cardiovascular disease (CVD) risk. MethodsA total of 714 randomly selected subjects from Northeastern Brazil were investigated in a cross-sectional study. Sociodemographic, anthropometric, and clinical data were recorded. Diagnostic test performance measures assessed the ability of the different MS definitions to identify those with pre-diabetes, T2DM and increased CVD risk. ResultsThe adjusted prevalence of MS was 36.1% applying the JIS criteria, 35.1% the IDF and 29.5% Modified NCEP. Women were more affected by MS according to all definitions. MS was significantly associated with pre-diabetes, T2DM and CVD risk following the three definitions. However, the JIS and IDF definitions showed higher sensitivity than the Modified NCEP to identify pre-diabetes, T2DM and CVD risk. The odds ratios for those conditions were not significantly different when comparing the definitions. ConclusionsMS is highly prevalent in Brazil, particularly among those with pre-diabetes, T2DM, and high CVD risk. The IDF and JIS criteria may be better suited in the Brazilian population to identify pre-diabetes, T2DM and CVD risk. This may also signify the importance of the assessment of MS in clinical practice.

Automated detection of pulmonary embolism in CT pulmonary angiograms using an AI-powered algorithm.

To evaluate the performance of an AI-powered algorithm for the automatic detection of pulmonary embolism (PE) on chest computed tomography pulmonary angiograms (CTPAs) on a large dataset. We retrospectively identified all CTPAs conducted at our institution in 2017 (n = 1499). Exams with clinical questions other than PE were excluded from the analysis (n = 34). The remaining exams were classified into positive (n = 232) and negative (n = 1233) for PE based on the final written reports, which defined the reference standard. The fully anonymized 1-mm series in soft tissue reconstructionserved as input for the PE detection prototype algorithm that was based on a deep convolutional neural network comprising a Resnet architecture. It was trained and validated on 28,000 CTPAs acquired at other institutions. The result series were reviewed using a web-based feedback platform. Measures of diagnostic performance were calculated on a per patient and a per finding level. The algorithm correctly identified 215 of 232 exams positive for pulmonary embolism (sensitivity 92.7%; 95% confidence interval [CI] 88.3-95.5%) and 1178 of 1233 exams negative for pulmonary embolism (specificity 95.5%; 95% CI 94.2-96.6%). On a per finding level, 1174 of 1352 findings marked as embolus by the algorithm were true emboli. Most of the false positive findings were due to contrast agent-related flow artifacts, pulmonary veins, and lymph nodes. The AI prototype algorithm we tested has a high degree of diagnostic accuracy for the detection of PE on CTPAs. Sensitivity and specificity are balanced, which is a prerequisite for its clinical usefulness. • An AI-based prototype algorithm showed a high degree of diagnostic accuracy for the detection of pulmonary embolism on CTPAs. • It can therefore help clinicians to automatically prioritize exams with a high suspection of pulmonary embolism and serve as secondary reading tool. • By complementing traditional ways of worklist prioritization in radiology departments, this can speed up the diagnostic and therapeutic workup of patients with pulmonary embolism and help to avoid false negative calls.

ECOG-ACRIN tomosynthesis mammographic imaging screening trial (EA1151).

TPS1597 Background: This randomized trial is intended to determine whether tomosynthesis (TM) should replace the current standard for breast cancer (BC) screening, digital mammography (DM). It is hypothesized that the population of women assigned TM screening for 3-5 rounds will have fewer advanced cancers than the population assigned to DM screening. Methods: 164,946 women, ages 45 to 74 years who present for screening mammography and consent to participate will be enrolled across 150 sites in the US, Canada and abroad. Women will be randomized to TM or DM. The frequency and number of screening examinations over a five year period will vary based on menopausal status and whether they have specific risk factors, including - hormone use, family history of BC, deleterious genes, prior benign breast biopsy with diagnosis of LCIS or atypia any kind, or dense breasts. Blood and buccal cells will be collected from as many enrolled women as are willing to provide the samples. All breast biopsies during the trial will undergo gene expression analysis for the PAM50 and other progression pathways (PAM50-plus). All subjects enrolled will be followed long term for at least eight years. The primary endpoint is the proportion of participants who have an advanced breast cancer diagnosed at any time within 4.5 years of randomization in to the trial. Secondary endpoints include measures of diagnostic and predictive performance; rates of recall, biopsy, and interval cancers, prevalence of breast cancer subtypes, and tumor subtype based on PAM50-plus analysis. As of January 17th 2020, there are 104 sites open and 21,452 women enrolled in the trial. The DSMC last reviewed the trial in June 2019 and suggested that the trial continue as planned. Clinical trial information: NCT03233191.

Diagnostic Performance of CT and Reverse Transcriptase Polymerase Chain Reaction for Coronavirus Disease 2019: A Meta-Analysis.

BackgroundRecent studies have suggested that chest computed tomography (CT) scans could be used as a primary screening or diagnostic tool for coronavirus disease 2019 (COVID-19) in epidemic areas.PurposeTo perform a meta-analysis to evaluate diagnostic performance measures, including predictive values, of chest CT and initial reverse transcriptase-polymerase chain reaction (RT-PCR).Materials and MethodsMEDLINE and Embase were searched from January 1, 2020 to April 3, 2020 for studies on COVID-19 that reported the sensitivity and/or specificity of CT scans and/or RT-PCR assays. The pooled sensitivity and specificity were estimated by using random-effects models. The actual prevalence (i.e., the proportion of confirmed patients among those tested) in eight countries was obtained from web sources, and the predictive values were calculated. Meta-regression was performed to reveal the effect of potential explanatory factors on the diagnostic performance measures.ResultsThe pooled sensitivity was 94% (95% CI: 91%, 96%; I2=95%) for chest CT and 89% (95% CI: 81%, 94%; I2=90%) for RT-PCR. The pooled specificity was 37% (95% CI: 26%, 50%; I2=83%) for chest CT. The prevalence of COVID-19 outside China ranged from 1.0% to 22.9%. For chest CT scans, the positive predictive value (PPV) ranged from 1.5% to 30.7%, and the negative predictive value (NPV) ranged from 95.4% to 99.8%. For RT-PCR,the PPV ranged from 47.3% to 96.4%, while the NPV ranged from 96.8% to 99.9%. The sensitivity of CT was affected by the distribution of disease severity, the proportion of patients with comorbidities, and the proportion of asymptomatic patients (all p < 0.05). The sensitivity of RT-PCR was negatively associated with the proportion of elderly patients (p = 0.01).ConclusionOutside of China where there is a low-prevalence of COVID-19 (1-22.9%), chest CT screening of patients with suspected disease had low positive predictive value (1.5-30.7%).

Differentiating Experience From Cue Utilization in Radiological Assessments.

This research was designed to examine the contribution of self-reported experience and cue utilization to diagnostic accuracy in the context of radiology. Within radiology, it is unclear how task-related experience contributes to the acquisition of associations between features with events in memory, or cues, and how they contribute to diagnostic performance. Data were collected from 18 trainees and 41 radiologists. The participants completed a radiology edition of the established cue utilization assessment tool EXPERTise 2.0, which provides a measure of cue utilization based on performance on a number of domain-specific tasks. The participants also completed a separate image interpretation task as an independent measure of diagnostic performance. Consistent with previous research, a k-means cluster analysis using the data from EXPERTise 2.0 delineated two groups, the pattern of centroids of which reflected higher and lower cue utilization. Controlling for years of experience, participants with higher cue utilization were more accurate on the image interpretation task compared to participants who demonstrated relatively lower cue utilization (p = .01). This study provides support for the role of cue utilization in assessments of radiology images among qualified radiologists. Importantly, it also demonstrates that cue utilization and self-reported years of experience as a radiologist make independent contributions to performance on the radiological diagnostic task. Task-related experience, including training, needs to be structured to ensure that learners have the opportunity to acquire feature-event relationships and internalize these associations in the form of cues in memory.

Abstract P527: Body Mass Index (bmi, Kg/m 2 ) vs. Body Fat % (bf%): Battle of Obesity Classification Metrics to Inform Underlying Cardiometabolic Risk

Introduction: The WHO defines overweight and obesity as abnormal or excessive fat accumulation that may impair health. However, for population and clinical classification of overweight and obesity BMI is most commonly used as a proxy for adiposity. Despite the strength of the association between higher BMI and cardiometabolic disease risk across populations, and minimal direct comparison of BMI v. BF%, BMI draws significant criticism as a metric to classify obesity and screen for cardiometabolic risk. Hypothesis: BF% defined obesity classification categories have higher underlying cardiometabolic risk compared to equivalent BMI categories. Methods: We analyzed data from CARDIA study participants from the year 20 exam with concurrent measures of BMI, and BF% measured by Dual-energy X-ray absorptiometry (DXA), plus clinical measures. Sex specific BF% obesity classification categories were defined by evidence-based diagnostic and classification performance measures vs. standard BMI cut points. Underlying cardiometabolic risk was defined by a continuous metabolic syndrome risk score (MetS), consisting of the averaged standardized deviates of the 5 ATP III components. Higher MetS z-scores indicate higher overall risk. MetS z-scores between categories (e.g. obesity defined by BMI v. BF%) were compared by z-tests. Results: The figure reports the distribution of obesity classification categories by the different metrics and underlying cardiometabolic risk. The MetS risk score was significantly higher in BMI defined overweight and obesity compared to BF% defined categories; and BF% defined obesity categories classify a greater prevalence of the population as obese compared to BMI. Results for MetS risk did not differ by race/sex or when including prevalent cardiometabolic disease. Conclusions: Physiologically defined BF% obesity categories classify a greater prevalence of obesity in CARDIA at year 20, whereas there was higher underlying cardiometabolic risk in BMI defined obesity classification categories.

Application of machine learning to ultrasound images to differentiate follicular neoplasms of the thyroid gland.

PurposeThis study was conducted to evaluate the diagnostic performance of machine learning in differentiating follicular adenoma from carcinoma using preoperative ultrasonography (US).MethodsIn this retrospective study, preoperative US images of 348 nodules from 340 patients were collected from two tertiary referral hospitals. Two experienced radiologists independently reviewed each image and categorized the nodules according to the 2015 American Thyroid Association guideline. Categorization of a nodule as highly suspicious was considered a positive diagnosis for malignancy. The nodules were manually segmented, and 96 radiomic features were extracted from each region of interest. Ten significant features were selected and used as final input variables in our in-house developed classifier models based on an artificial neural network (ANN) and support vector machine (SVM). The diagnostic performance of radiologists and both classifier models was calculated and compared.ResultsIn total, 252 nodules from 245 patients were confirmed as follicular adenoma and 96 nodules from 95 patients were diagnosed as follicular carcinoma. As measures of diagnostic performance, the average sensitivity, specificity, and accuracy of the two experienced radiologists in discriminating follicular adenoma from carcinoma on preoperative US images were 24.0%, 84.0%, and 64.8%, respectively. The sensitivity, specificity, and accuracy of the ANN and SVM-based models were 32.3%, 90.1%, and 74.1% and 41.7%, 79.4%, and 69.0%, respectively. The kappa value of the two radiologists was 0.076, corresponding to slight agreement.ConclusionMachine learning-based classifier models may aid in discriminating follicular adenoma from carcinoma using preoperative US.

6. Audit of breast screening core biopsy diagnostic performance measures at pathwest fiona stanley hospital (FSH) from 2015 to 2016

Background: Core needle biopsy (CNB) is widely utilised in the investigation of screen detected breast abnormalities. National and international guidelines recommend classification of biopsies into categories: ‘non-diagnostic (B1)’, ‘benign (B2)’, ‘atypical/indeterminate (B3)’, ‘suspicious (B4)’ and ‘malignant (B5)’, each with common diagnostic and therapeutic implications. Accurate classification is imperative to reduce morbidity associated with breast screening programs such as over-diagnosis and the resulting overtreatment of patients. This issue is particularly relevant to the ‘indeterminate/atypical (B3)’ and ‘suspicious (B4)’ lesions on core biopsy where further tissue is required specifically for the purpose of diagnostic clarification and not treatment per se. Aims: To assess the performance of Fiona Stanley Hospital (FSH) PathWest category reporting of screening breast core biopsies over a 2-year period against UK National Health Service (NHS) standards. Methods: Retrospective collection of data regarding the diagnostic category classification of all breast core biopsies reported at PathWest FSH in the period between January 2015 and December 2016. Results and conclusions: Out of 2470 breast lesions examined in this audit, 3.2% were non-diagnostic, 55.7% benign, 6.1% atypical, 0.4% suspicious and 34.5% malignant. The rate of B3 and B4 lesions in this audit were well within the NHS audit expected targets. The other categories were also comparable to rates found in similar large international studies. These results, a component of the pathology quality assurance metrics of a breast screening program, are a surrogate for adequate application of diagnostic criteria to all diagnostic categories, but are particularly relevant to the atypical/indeterminate category where there is great potential for over-investigation.