Alternative Free-response Receiver Operating Characteristic Research Articles

MP2RAGE vs. MPRAGE surface-based morphometry in focal epilepsy.

In drug-resistant focal epilepsy, detecting epileptogenic lesions using MRI poses a critical diagnostic challenge. Here, we assessed the utility of MP2RAGE-a T1-weighted sequence with self-bias correcting properties commonly utilized in ultra-high field MRI-for the detection of epileptogenic lesions using a surface-based morphometry pipeline based on FreeSurfer, and compared it to the common approach using T1w MPRAGE, both at 3T. We included data from 32 patients with focal epilepsy (5 MRI-positive, 27 MRI-negative with lobar seizure onset hypotheses) and 94 healthy controls from two epilepsy centres. Surface-based morphological measures and intensities were extracted and evaluated in univariate GLM analyses as well as multivariate unsupervised 'novelty detection' machine learning procedures. The resulting prediction maps were analyzed over a range of possible thresholds using alternative free-response receiver operating characteristic (AFROC) methodology with respect to the concordance with predefined lesion labels or hypotheses on epileptogenic zone location. We found that MP2RAGE performs at least comparable to MPRAGE and that especially analysis of MP2RAGE image intensities may provide additional diagnostic information. Secondly, we demonstrate that unsupervised novelty-detection machine learning approaches may be useful for the detection of epileptogenic lesions (maximum AFROC AUC 0.58) when there is only a limited lesional training set available. Third, we propose a statistical method of assessing lesion localization performance in MRI-negative patients with lobar hypotheses of the epileptogenic zone based on simulation of a random guessing process as null hypothesis. Based on our findings, it appears worthwhile to study similar surface-based morphometry approaches in ultra-high field MRI (≥ 7 T).

Development and multicenter validation of deep convolutional neural network-based detection of colorectal cancer on abdominal CT.

This study aims to develop computer-aided detection (CAD) for colorectal cancer (CRC) using abdominal CT based on a deep convolutional neural network. This retrospective study included consecutive patients with colorectal adenocarcinoma who underwent abdominal CT before CRC resection surgery (training set = 379, test set = 103). We customized the 3D U-Net of nnU-Net (CUNET) for CRC detection, which was trained with fivefold cross-validation using annotated CT images. CUNET was validated using datasets covering various clinical situations and institutions: an internal test set (n = 103), internal patients with CRC first determined by CT (n = 54) and asymptomatic CRC (n = 51), and an external validation set from two institutions (n = 60). During each validation, data from the healthy population were added (internal = 60; external = 130). CUNET was compared with other deep CNNs: residual U-Net and EfficientDet. The CAD performances were evaluated using per-CRC sensitivity (true positive/all CRCs), free-response receiver operating characteristic (FROC), and jackknife alternative FROC (JAFROC) curves. CUNET showed a higher maximum per-CRC sensitivity than residual U-Net and EfficientDet (internal test set 91.3% vs. 61.2%, and 64.1%). The per-CRC sensitivity of CUNET at false-positive rates of 3.0 was as follows: internal CRC determined by CT, 89.3%; internal asymptomatic CRC, 87.3%; and external validation, 89.6%. CUNET detected 69.2% (9/13) of CRCs missed by radiologists and 89.7% (252/281) of CRCs from all validation sets. CUNET can detect CRC on abdominal CT in patients with various clinical situations and from external institutions. • Customized 3D U-Net of nnU-Net (CUNET) can be applied to the opportunistic detection of colorectal cancer (CRC) in abdominal CT, helping radiologists detect unexpected CRC. • CUNET showed the best performance at false-positive rates ≥ 3.0, and 30.1% of false-positives were in the colorectum. CUNET detected 69.2% (9/13) of CRCs missed by radiologists and 87.3% (48/55) of asymptomatic CRCs. • CUNET detected CRCs in multiple validation sets composed of varying clinical situations and from different institutions, and CUNET detected 89.7% (252/281) of CRCs from all validation sets.

Automated deep-learning system in the assessment of MRI-visible prostate cancer: comparison of advanced zoomed diffusion-weighted imaging and conventional technique

BackgroundDeep-learning-based computer-aided diagnosis (DL-CAD) systems using MRI for prostate cancer (PCa) detection have demonstrated good performance. Nevertheless, DL-CAD systems are vulnerable to high heterogeneities in DWI, which can interfere with DL-CAD assessments and impair performance. This study aims to compare PCa detection of DL-CAD between zoomed-field-of-view echo-planar DWI (z-DWI) and full-field-of-view DWI (f-DWI) and find the risk factors affecting DL-CAD diagnostic efficiency.MethodsThis retrospective study enrolled 354 consecutive participants who underwent MRI including T2WI, f-DWI, and z-DWI because of clinically suspected PCa. A DL-CAD was used to compare the performance of f-DWI and z-DWI both on a patient level and lesion level. We used the area under the curve (AUC) of receiver operating characteristics analysis and alternative free-response receiver operating characteristics analysis to compare the performances of DL-CAD using f- DWI and z-DWI. The risk factors affecting the DL-CAD were analyzed using logistic regression analyses. P values less than 0.05 were considered statistically significant.ResultsDL-CAD with z-DWI had a significantly better overall accuracy than that with f-DWI both on patient level and lesion level (AUCpatient: 0.89 vs. 0.86; AUClesion: 0.86 vs. 0.76; P < .001). The contrast-to-noise ratio (CNR) of lesions in DWI was an independent risk factor of false positives (odds ratio [OR] = 1.12; P < .001). Rectal susceptibility artifacts, lesion diameter, and apparent diffusion coefficients (ADC) were independent risk factors of both false positives (ORrectal susceptibility artifact = 5.46; ORdiameter, = 1.12; ORADC = 0.998; all P < .001) and false negatives (ORrectal susceptibility artifact = 3.31; ORdiameter = 0.82; ORADC = 1.007; all P ≤ .03) of DL-CAD.ConclusionsZ-DWI has potential to improve the detection performance of a prostate MRI based DL-CAD.Trial registrationChiCTR, NO. ChiCTR2100041834. Registered 7 January 2021.

Added value of contrast enhancement boost images in routine multiphasic contrast-enhanced CT for the diagnosis of small (

PurposeTo investigate the added value of contrast enhancement boost (CE-boost) images in multiphasic contrast-enhanced CT (CE-CT) for diagnosing small (<20 mm) hypervascular hepatocellular carcinoma (HCC). Materials and methodsThis retrospective study included 69 patients (age, 74 ± 8 years; 52 men) with 70 hypervascular HCCs (<20 mm) who underwent multiphasic CE-CT (pre-contrast, late arterial phase [LAP], portal venous phase [PVP], and equilibrium phase). Two types of CE-boost images were generated by subtracting PVP from LAP (LA-PV) images and LAP from PVP (PV-LA) images to enhance the contrast effect of hepatic arterial and portal venous perfusion more selectively. Tumor-to-liver contrast-to-noise ratios (CNRs) in CE-boost images were compared with those in CE-CT images using the Wilcoxon signed-rank test. Two independent readers reviewed the imaging datasets: CE-CT alone and CE-CT with CE-boost images. The diagnostic performance of each dataset was compared using jackknife alternative free-response receiver operating characteristics (JAFROC-1). ResultsThe tumor-to-liver CNRs in the LA-PV (6.4 ± 3.0) and PV-LA (−3.3 ± 2.1) images were greater than those in the LAP (3.2 ± 1.7) and PVP images (−1.1 ± 1.4) (p <.001 for both). The reader-averaged figures of merit were 0.751 for CE-CT alone and 0.807 for CE-CT with CE-boost images (p <.001). Sensitivities increased by adding CE-boost images for both readers (p <.001 and = 0.03), while positive predictive values were equivalent (p >.99). ConclusionAdding CE-boost images to multiphasic CE-CT can improve the diagnostic accuracy and sensitivity for small hypervascular HCC by increasing the tumor-to-liver CNR.

Multiarterial Phase Acquisition in Gadoxetic Acid-Enhanced Liver MRI for the Detection of Hypervascular Hepatocellular Carcinoma in High-Risk Patients: Comparison of Compressed Sensing Versus View Sharing Techniques.

The aim of this study was to compare compressed sensing (CS) and view sharing (VS) techniques for single breath-hold multiarterial phase imaging with respect to image quality and focal liver observation detectability during gadoxetic acid-enhanced magnetic resonance imaging in patients at high risk for hepatocellular carcinoma (HCC). A total of 385 patients who underwent gadoxetic acid-enhanced magnetic resonance imaging, including triple arterial phases using either CS (n = 224) or VS (n = 161) techniques, were retrospectively included. Among them, 117 patients had 171 focal liver observations (median diameter, 1.3 cm), which were classified according to Liver Imaging Reporting and Data System version 2018. The acquisition rate of optimally timed late arterial phase (LAP) was assessed, and image quality, including respiratory motion artifact and observation conspicuity, was rated on a 4-point scale by 3 radiologists. The Mann-Whitney U test and nonparametric test for repeated measures data were used for image quality and observation conspicuity analysis. The jackknife alternative free-response receiver operating characteristics method was used to compare the observation detectability between the 2 techniques. The CS technique showed significantly higher acquisition rate of optimally timed LAP without transient severe motion (82.1% [184/224] vs 71.4% [115/161]; P = 0.013) than the VS technique. The CS technique also demonstrated significantly improved overall image quality (3.42 ± 0.70 vs 2.97 ± 0.61; P < 0.001) compared with the VS technique. Regarding the detection of hyperenhancing observations, there was no significant difference between the figure of merits of CS and VS techniques (0.660 vs 0.665; P = 0.890). However, the CS technique showed a higher detection rate in Liver Imaging Reporting and Data System M (LR-M, probably or definitely malignant but not HCC specific) observations than the VS technique (100.0% [9/9] vs 44.4% [8/18]; P = 0.009). The CS technique tended to provide optimally timed LAP without transient severe motion and demonstrated greater detection rate of LR-M observations than the VS technique in patients at high risk of HCC.

Detection of distant metastases and distant second primary cancers in head and neck squamous cell carcinoma: comparison of [18F]FDG PET/MRI and [18F]FDG PET/CT

PurposeThis prospective study aimed to compare the diagnostic performance of [18]FDG PET/MRI and PET/CT for the detection of distant metastases and distant second primary cancers in patients with head and neck squamous cell carcinoma (HNSCC).MethodsA total of 103 [18F]FDG PET/MRI examinations immediately followed by PET/CT were obtained in 82 consecutive patients for staging of primary HNSCC (n = 38), suspected loco-regional recurrence/follow-up (n = 41) or unknown primary HNSCC (n = 3). Histology and follow-up > 2 years formed the standard of reference. Blinded readers evaluated the anonymized PET/MRI and PET/CT examinations separately using a 5-point Likert score. Statistical analysis included: receiver operating characteristic (ROC) analysis, jackknife alternative free-response ROC (JAFROC) and region-of-interest (ROI)-based ROC to account for data clustering and sensitivity/specificity/accuracy comparisons for a score ≥ 3.ResultsDistant metastases and distant second primary cancers were present in 23/103 (22%) examinations in 16/82 (19.5%) patients, and they were more common in the post-treatment group (11/41, 27%) than in the primary HNSCC group (3/38, 8%), p = 0.039. The area under the curve (AUC) per patient/examination/lesion was 0.947 [0.927–1]/0.965 [0.917–1]/0.957 [0.928–0.987] for PET/MRI and 0.975 [0.950–1]/0.968 [0.920–1]/0.944 [0.910–0.979] for PET/CT, respectively (p > 0.05). The diagnostic performance of PET/MRI and PET/CT was similar according to JAFROC (p = 0.919) and ROI-based ROC analysis (p = 0.574). Sensitivity/specificity/accuracy for PET/MRI and PET/CT for a score ≥ 3 was 94%/88%/89% and 94%/91%/91% per patient, 96%/90%/91% and 96%/93%/93% per examination and 95%/85%/90% and 90%/86%/88% per lesion, respectively, p > 0.05.ConclusionsIn HNSCC patients, PET/MRI and PET/CT had a high and similar diagnostic performance for detecting distant metastases and distant second primary cancers.

Rethinking Annotation Granularity for Overcoming Shortcuts in Deep Learning-based Radiograph Diagnosis: A Multicenter Study.

To evaluate the ability of fine-grained annotations to overcome shortcut learning in deep learning (DL)-based diagnosis using chest radiographs. Two DL models were developed using radiograph-level annotations (disease present: yes or no) and fine-grained lesion-level annotations (lesion bounding boxes), respectively named CheXNet and CheXDet. A total of 34 501 chest radiographs obtained from January 2005 to September 2019 were retrospectively collected and annotated regarding cardiomegaly, pleural effusion, mass, nodule, pneumonia, pneumothorax, tuberculosis, fracture, and aortic calcification. The internal classification performance and lesion localization performance of the models were compared on a testing set (n = 2922); external classification performance was compared on National Institutes of Health (NIH) Google (n = 4376) and PadChest (n = 24 536) datasets; and external lesion localization performance was compared on the NIH ChestX-ray14 dataset (n = 880). The models were also compared with radiologist performance on a subset of the internal testing set (n = 496). Performance was evaluated using receiver operating characteristic (ROC) curve analysis. Given sufficient training data, both models performed similarly to radiologists. CheXDet achieved significant improvement for external classification, such as classifying fracture on NIH Google (CheXDet area under the ROC curve [AUC], 0.67; CheXNet AUC, 0.51; P < .001) and PadChest (CheXDet AUC, 0.78; CheXNet AUC, 0.55; P < .001). CheXDet achieved higher lesion detection performance than CheXNet for most abnormalities on all datasets, such as detecting pneumothorax on the internal set (CheXDet jackknife alternative free-response ROC [JAFROC] figure of merit [FOM], 0.87; CheXNet JAFROC FOM, 0.13; P < .001) and NIH ChestX-ray14 (CheXDet JAFROC FOM, 0.55; CheXNet JAFROC FOM, 0.04; P < .001). Fine-grained annotations overcame shortcut learning and enabled DL models to identify correct lesion patterns, improving the generalizability of the models.Keywords: Computer-aided Diagnosis, Conventional Radiography, Convolutional Neural Network (CNN), Deep Learning Algorithms, Machine Learning Algorithms, Localization Supplemental material is available for this article © RSNA, 2022.

Abbreviated Gadoxetic Acid-Enhanced MRI for the Detection of Liver Metastases in Patients With Potentially Resectable Pancreatic Ductal Adenocarcinoma.

Gadoxetic acid-enhanced magnetic resonance imaging (MRI) is useful in detecting liver metastases from pancreatic ductal adenocarcinoma (PDAC). However, the long examination time limits its utility in the initial workup of patients with PDAC. To evaluate the incremental value of an abbreviated gadoxetic acid-enhanced MRI for the detection of liver metastases in patients with PDAC. Retrospective. Patients (N=130) with potentially resectable PDAC (women, 58 [44.6%]). 1.5 T and 3T; gradient dual-echo T1-weighted (in-phase and opposed-phase), fat-suppressed fast spin-echo T2-weighted, single-shot echo-planar diffusion-weighted, and three-dimensional fat-suppressed T1-weighted gradient-echo dynamic contrast-enhanced and hepatobiliary phase sequences, as well as contrast-enhanced computed tomography (CECT). Three radiologists independently reviewed three different image sets to detect liver metastases: set 1, CECT alone; set 2, CECT and abbreviated MRI comprising fat-suppressed T2-weighted, diffusion-weighted, and hepatobiliary phase images; and set 3, CECT and standard gadoxetic acid-enhanced MRI. Figure of merit (FOM) was compared using the jackknife alternative free-response receiver operating characteristics, and other per-lesion and per-patient diagnostic parameters for each image set were compared using McNemar's and Fisher's test. P < 0.05 was considered statistically significant. A total of 43 liver metastases were identified in 13 patients. Reader-averaged FOM to detect liver metastases were significantly higher for sets 2 (0.884) and 3 (0.886) than for set 1 (0.609), while they were comparable between sets 2 and 3 (P=0.96). The mean per-patient sensitivities, negative predictive values, and accuracies were significantly higher for sets 2 and 3 than for set 1, while those between sets 2 and 3 were not significantly different (not applicable, P > 0.99, and P > 0.99, respectively). Gadoxetic acid-enhanced MRI combined with CECT had higher diagnostic performance than CECT alone for the detection of liver metastases in patients with PDAC. The incremental values were comparable for the abbreviated MRI and standard MRI. 3 TECHNICAL EFFICACY: Stage 2.

Bone Suppression on Chest Radiographs for Pulmonary Nodule Detection: Comparison between a Generative Adversarial Network and Dual-Energy Subtraction

ObjectiveTo compare the effects of bone suppression imaging using deep learning (BSp-DL) based on a generative adversarial network (GAN) and bone subtraction imaging using a dual energy technique (BSt-DE) on radiologists’ performance for pulmonary nodule detection on chest radiographs (CXRs).Materials and MethodsA total of 111 adults, including 49 patients with 83 pulmonary nodules, who underwent both CXR using the dual energy technique and chest CT, were enrolled. Using CT as a reference, two independent radiologists evaluated CXR images for the presence or absence of pulmonary nodules in three reading sessions (standard CXR, BSt-DE CXR, and BSp-DL CXR). Person-wise and nodule-wise performances were assessed using receiver-operating characteristic (ROC) and alternative free-response ROC (AFROC) curve analyses, respectively. Subgroup analyses based on nodule size, location, and the presence of overlapping bones were performed.ResultsBSt-DE with an area under the AFROC curve (AUAFROC) of 0.996 and 0.976 for readers 1 and 2, respectively, and BSp-DL with AUAFROC of 0.981 and 0.958, respectively, showed better nodule-wise performance than standard CXR (AUAFROC of 0.907 and 0.808, respectively; p ≤ 0.005). In the person-wise analysis, BSp-DL with an area under the ROC curve (AUROC) of 0.984 and 0.931 for readers 1 and 2, respectively, showed better performance than standard CXR (AUROC of 0.915 and 0.798, respectively; p ≤ 0.011) and comparable performance to BSt-DE (AUROC of 0.988 and 0.974; p ≥ 0.064). BSt-DE and BSp-DL were superior to standard CXR for detecting nodules overlapping with bones (p < 0.017) or in the upper/middle lung zone (p < 0.017). BSt-DE was superior (p < 0.017) to BSp-DL in detecting peripheral and sub-centimeter nodules.ConclusionBSp-DL (GAN-based bone suppression) showed comparable performance to BSt-DE and can improve radiologists’ performance in detecting pulmonary nodules on CXRs. Nevertheless, for better delineation of small and peripheral nodules, further technical improvements are required.

Comparison of two-view versus single-view digital breast tomosynthesis and 2D-mammography in breast cancer surveillance imaging.

PurposeLimited work has been performed for the implementation of digital breast tomosynthesis (DBT) in breast cancer surveillance imaging. The aim of this study was to investigate the differences between two different DBT implementations in breast cancer surveillance imaging, for patients with a personal history of breast cancer.MethodThe DBT implementations investigated were: (1) 2-view 2D digital mammography and 2-view DBT (2vDM&2vDBT) (2) 1-view (cranial-caudal) DM and 1-view (mediolateral-oblique) DBT (1vDM&1vDBT). Clinical performance of these two implementations was assessed retrospectively using observer studies with 118 sets of real patient images, from a single imaging centre, and six observers. Sensitivity, specificity and area under the curve (AUC) using the Jack-knife alternative free-response receiver operating characteristics (JAFROC) analysis were evaluated.ResultsResults suggest that the two DBT implementations are not significantly different in terms of sensitivity, specificity and AUC. When looking at the two main different lesion types, non-calcifications and calcifications, and two different density levels, no difference in the performance of the two DBT implementations was found.ConclusionsSince 1vDM&1vDBT exposes the patient to half the dose of 2vDM&2vDBT, it might be worth considering 1vDM&1vDBT in breast cancer surveillance imaging. However, larger studies are required to conclude on this matter.

Improving radiologist's ability in identifying particular abnormal lesions on mammograms through training test set with immediate feedback

It has been shown that there are differences in diagnostic accuracy of cancer detection on mammograms, from below 50% in developing countries to over 80% in developed world. One previous study reported that radiologists from a population in Asia displayed a low mammographic cancer detection of 48% compared with over 80% in developed countries, and more importantly, that most lesions missed by these radiologists were spiculated masses or stellate lesions. The aim of this study was to explore the performance of radiologists after undertaking a training test set which had been designed to improve the capability in detecting a specific type of cancers on mammograms. Twenty-five radiologists read two sets of 60 mammograms in a standardized mammogram reading room. The first test set focused on stellate or spiculated masses. When radiologists completed the first set, the system displayed immediate feedback to the readers comparing their performances in each case with the truth of cancer cases and cancer types so that the readers could identify individual-based errors. Later radiologists were asked to read the second set of mammograms which contained different types of cancers including stellate/spiculated masses, asymmetric density, calcification, discrete mass and architectural distortion. Case sensitivity, lesion sensitivity, specificity, receiver operating characteristics (ROC) and Jackknife alternative free-response receiver operating characteristics (JAFROC) were calculated for each participant and their diagnostic accuracy was compared between two sessions. Results showed significant improvement among radiologists in case sensitivity (+ 11.4%; P < 0.05), lesion sensitivity (+ 18.7%; P < 0.01) and JAFROC (+ 11%; P < 0.01) in the second set compared with the first set. The increase in diagnostic accuracy was also recorded in the detection of stellate/spiculated mass (+ 20.6%; P < 0.05). This indicated that the performance of radiologists in detecting malignant lesions on mammograms can be improved if an appropriate training intervention is applied after the readers’ weakness and strength are identified.

Evaluation of an automatic method for detection of defects in linear and curvilinear ultrasound transducers.

The high incidence of defective ultrasound transducers in clinical practice has been shown in several studies. Recently, a novel method using only stored images for automatic detection of defective transducers was presented. The method makes it possible to remotely monitor many transducers at the same time and send a notification when a defective transducer is found. The purpose of the present study was to evaluate the novel method and assess how well it performs when compared to an established method as reference. To evaluate the novel method, in-air images were collected from 81 transducers in radiologic departments in nine hospitals. Two observers assessed the in-air images and marked the defects. Receiver operating characteristic (ROC)- and alternative free response receiver operating characteristic (AFROC)-curves and their figures of merit (FOM) were calculated for the novel method, using marked defects in the in-air images as reference truth. The area under the ROC curve was 0.88 (SD 0.06), and the AFROC FOM was 0.71 (SE 0.07). The result shows that the novel method has a good agreement with the in-air method for detecting defects in ultrasound systems. This indicates that the novel method could be a complement to the normal quality control for early, and automatic detection of defects.

Deep Learning Algorithm for Chest Radiography: Multicenter Respiratory Outpatient Diagnostic Cohort Study

Background: Commercially available deep learning algorithms (DLAs) for chest radiography (CXR) have been evaluated with variable clinical settings. This study evaluated a DLA for detecting and localizing referable thoracic abnormalities on CXRs in a multicenter respiratory outpatient diagnostic cohort and compared the performance of physicians with and without DLA assistance. Methods: A total of 6,006 consecutive participants who visited respiratory outpatient clinics from 3 institutions and underwent both CXR and CT in 2018 were retrospectively evaluated with DLA. Observer performance tests were conducted with a subset of 230 randomly sampled participants to assess whether the algorithm enhanced physician performance. The area under the receiver operating characteristic (ROC) curve (AUC) and the area under the alternative free-response ROC (AUAFROC) were measured to evaluate the performance of the DLA and physicians in image classification and lesion localization, respectively. Findings: In the assessments performed in 6,006 participants, 4,264 referable thoracic abnormal lesions were found in 3,337 (55·6%) participants, including 1,173 pulmonary nodules/masses (20%), 919 consolidations (15%), 15 pneumothoraces (0·2%), and 2,157 other lesions (51%). DLA showed an AUC of 0·86 to 0·87, sensitivity of 0·87 to 0·90, specificity of 0·70 to 0·76, false-positive findings per image of 0·31 to 0·42, and AUAFROC of 0·86 to 0·87. For the randomly sampled dataset (n=230), the average AUC increased from 0·86 (95% CI: 0·82, 0·90) to 0·89 (95% CI: 0·85, 0·92) (P=0·003) and the average AUFROC increased from 0·80 (95% CI: 0·76, 0·84) to 0·82 (95% CI: 0·78, 0·86) (P=0·003) with the aid of DLA. The AUROC of standalone DLA was 0·91, which was significantly higher than the standalone AUROCs of pulmonologists (0·84,P=0·01) and radiology residents (0·85, P=0·03). Interpretation: The DLA showed acceptable performance and could enhance physician performance for diagnosing referable thoracic abnormalities in daily clinical practice. Funding Statement: This work was supported by grant (HI19C0847) from Korea Health Industry Development Institute. Declaration of Interests: Activities not related to the present article: KNJ and HK received a research grant from Lunit. Other authors have neither financial interests nor conflicts of interest. Ethics Approval Statement: This study was approved by the institutional review boards of all participating institutions, which waived the requirement for patient consent.

Impact of Hours Awake and Hours Slept at Night on Radiologists' Mammogram Interpretations.

To examine whether radiologists' mammogram reading performance varies according to how long they have been awake ("hours awake") and the number of hours they slept ("hours slept") the night before a reading session. Retrospective data were retrieved from the BreastScreen Reader Assessment Strategy database. Malignancy-enriched mammographic readings were performed by 133 radiologists. Information on their hours awake and hours slept was collected. Analysis of covariance was performed to determine whether these two variables influenced radiologists' sensitivity, specificity, lesion sensitivity, receiver operating characteristic (ROC) curve, and jackknife alternative free-response ROC. Radiologists were divided into a more experienced and a less experienced groups (based on reading ≥2,000 and <2,000 mammogram readings per year, respectively). The hours awake significantly influenced less experienced radiologists' lesion sensitivity (F6,63= 2.51; P= .03). Those awake for <2 hours had significantly lower lesion sensitivity than those awake for 8 to 10 hours (P= .01), and those awake for 4 to 6 hours had significantly lower lesion sensitivity than those awake for 8 to 10 hours (P= .002) and 10 to 12 hours (P= .02). The hours slept also influenced the ROC values of less experienced radiologists (F1,68= 4.96; P= .02). Radiologists with up to 6 hours of sleep had a significantly lower value (0.72) than those who had slept more than 6 hours (0.77). No statistically significant findings were noted for more experienced radiologists. Inexperienced radiologists' performance may be influenced by the hours awake and hours slept before reading sessions.

Multi-professional image interpretation: Performance in preliminary clinical evaluation of appendicular radiographs

IntroductionThe study evaluated the performance of a cohort of healthcare professionals in image interpretation of appendicular radiographic examinations following a preliminary clinical evaluation (PCE) course. MethodsA systematic review and meta-analysis of image interpretation performance by non-medical staff was undertaken. Thirty three participants (seven radiographers, nineteen nurses, five paramedics, and two physiotherapists) completed an image based Objective Structured Examination (OSE) following a short course on PCE of appendicular radiographs. The case bank comprised 25 appendicular radiographic examinations. Prevalence of abnormal examinations approximated 52% (traumatic conditions). The OSE test scores were analysed with Alternative Free-response Receiver Operating Characteristic (AFROC) calculation of Area Under the Curve (AUC), sensitivity, specificity, and Cohen's Kappa (k) for multi-reader agreement. Professional subgroup results were compared with Analysis of Variance (ANOVA), and compared to the literature review. ResultsThe participant subgroup results for radiographers were sensitivity 79.7%, specificity 92.9%, k = 72; AUC 86.5. Nurses sensitivity 76.2%, specificity 80.4%, k = 56; AUC 78.3. Paramedics sensitivity 80.3%, specificity 81.6%, k = 61; AUC 81.5. Physiotherapists sensitivity 90.4%, specificity 91.6%, k = 82; AUC 91.8. ANOVA Samples scores (f = 1.5; p = 0.23), sensitivity (f = 0.5; p = 0.65), specificity (f = 2.8; p = 0.05). The comparison of the radiographers and nurses results to the published literature were encouraging (SROC AUC 93.1 and 91.2, respectively). The paramedics and physiotherapists results could not be compared to published standards or abilities on the task due to a lack of literature available. ConclusionThe study results of image interpretation and commentary of appendicular radiographs in an academic environment by this multi-professional sample were to a high standard, further work is recommended on a larger sample.

High Acceleration Three-Dimensional T1-Weighted Dual Echo Dixon Hepatobiliary Phase Imaging Using Compressed Sensing-Sensitivity Encoding: Comparison of Image Quality and Solid Lesion Detectability with the Standard T1-Weighted Sequence.

ObjectiveTo compare a high acceleration three-dimensional (3D) T1-weighted gradient-recalled-echo (GRE) sequence using the combined compressed sensing (CS)-sensitivity encoding (SENSE) method with a conventional 3D GRE sequence using SENSE, with respect to image quality and detectability of solid focal liver lesions (FLLs) in the hepatobiliary phase (HBP) of gadoxetic acid-enhanced liver MRI.Materials and MethodsA total of 217 patients with gadoxetic acid-enhanced liver MRI at 3T (54 in the preliminary study and 163 in the main study) were retrospectively included. In the main study, HBP imaging was done twice using the standard mDixon-3D-GRE technique with SENSE (acceleration factor [AF]: 2.8, standard mDixon-GRE) and the high acceleration mDixon-3D GRE technique using the combined CS-SENSE technique (CS-SENSE mDixon-GRE). Two abdominal radiologists assessed the two MRI data sets for image quality in consensus. Three other abdominal radiologists independently assessed the diagnostic performance of each data set and its ability to detect solid FLLs in 117 patients with 193 solid nodules and compared them using jackknife alternative free-response receiver operating characteristics (JAFROC).ResultsThere was no significant difference in the overall image quality. CS-SENSE mDixon-GRE showed higher image noise, but lesser motion artifact levels compared with the standard mDixon-GRE (all p < 0.05). In terms of lesion detection, reader-averaged figures-of-merit estimated with JAFROC was 0.918 for standard mDixon-GRE, and 0.953 for CS-SENSE mDixon-GRE (p = 0.142). The non-inferiority of CS-SENSE mDixon-GRE over standard mDixon-GRE was confirmed (difference: 0.064 [−0.012, 0.081]).ConclusionThe CS-SENSE mDixon-GRE HBP sequence provided comparable overall image quality and non-inferior solid FFL detectability compared with the standard mDixon-GRE sequence, with reduced acquisition time.

Development and Validation of a Deep Learning-Based Automated Detection Algorithm for Major Thoracic Diseases on Chest Radiographs

Background: Interpretation of chest radiographs (CRs) is a challenging task prone to errors, requiring expert readers. An automated system that can accurately classify CRs may help streamline the clinical workflow. Methods: We developed a deep-learning-based automatic detection (DLAD) algorithm for classification of CRs with major thoracic diseases including pulmonary malignancy, active pulmonary tuberculosis, pneumonia, and pneumothorax, using 54,221 normal CRs and 47,917 abnormal CRs, which were labeled and annotated by board certified radiologists. Performance of DLAD was validated using five independent datasets from different institutions. To compare the performances of DLAD with physicians, an observer performance test was conducted by 15 physicians including non-radiology physicians, board-certified radiologists, and thoracic radiologists. Image-wise classification performances and lesion-wise localization performances were measured by area under the receiver operating characteristic (ROC) curves, and area under the alternative free-response ROC curves, respectively. Findings: DLAD demonstrated classification performance of 0*973-1*000 and localization performance of 0*923-1*000 in independent datasets. DLAD demonstrated significantly higher performance than all three physician groups in both image-wise classification (0*983 vs. 0*814-0*932) and lesion-wise localization (0*985 vs. 0*781-0*907). Significant improvements in both image-wise classification (performance increment, 0*026-0*090) and lesion-wise localization (performance increment, 0*031-0*092) were observed in all three physician groups with assistance of the algorithm. Interpretation: Our DLAD demonstrated excellent and consistent performance in the detection of major thoracic diseases on CR, outperforming even thoracic radiologists. Funding Statement: The present study was supported by the Seoul National University Hospital Research fund (grant number: 04- 2016-3000), Lunit Inc. and the Seoul Research & Buisiness Development Program (grant number: FI170002). Declaration of Interests: Chang Min Park reports grants from Seoul National University Hospital, Seoul Metropolitan Government, and Lunit Inc., during the conduct of the study; Sunggyun Park and Jaehong Aum are employees of Lunit Inc. Ethics Approval Statement: The present study was approved by the institutional review boards of all participating institutions. Patients’ informed consents were waived.

Development and Validation of a Deep Learning-based Automatic Detection Algorithm for Active Pulmonary Tuberculosis on Chest Radiographs.

BackgroundDetection of active pulmonary tuberculosis on chest radiographs (CRs) is critical for the diagnosis and screening of tuberculosis. An automated system may help streamline the tuberculosis screening process and improve diagnostic performance.MethodsWe developed a deep learning–based automatic detection (DLAD) algorithm using 54c221 normal CRs and 6768 CRs with active pulmonary tuberculosis that were labeled and annotated by 13 board-certified radiologists. The performance of DLAD was validated using 6 external multicenter, multinational datasets. To compare the performances of DLAD with physicians, an observer performance test was conducted by 15 physicians including nonradiology physicians, board-certified radiologists, and thoracic radiologists. Image-wise classification and lesion-wise localization performances were measured using area under the receiver operating characteristic (ROC) curves and area under the alternative free-response ROC curves, respectively. Sensitivities and specificities of DLAD were calculated using 2 cutoffs (high sensitivity [98%] and high specificity [98%]) obtained through in-house validation.ResultsDLAD demonstrated classification performance of 0.977–1.000 and localization performance of 0.973–1.000. Sensitivities and specificities for classification were 94.3%–100% and 91.1%–100% using the high-sensitivity cutoff and 84.1%–99.0% and 99.1%–100% using the high-specificity cutoff. DLAD showed significantly higher performance in both classification (0.993 vs 0.746–0.971) and localization (0.993 vs 0.664–0.925) compared to all groups of physicians.ConclusionsOur DLAD demonstrated excellent and consistent performance in the detection of active pulmonary tuberculosis on CR, outperforming physicians, including thoracic radiologists.

Chest X-ray Interpretation by Radiographers Is Not Inferior to Radiologists: A Multireader, Multicase Comparison Using JAFROC (Jack-knife Alternative Free-response Receiver Operating Characteristics) Analysis

Chest X-rays (CXR) are one of the most frequently requested imaging examinations and are fundamental to many patient pathways. The aim of this study was to investigate the diagnostic accuracy of CXR interpretation by reporting radiographers (technologists). A cohort of consultant radiologists (n = 10) and reporting radiographers (technologists; n = 11) interpreted a bank (n = 106) of adult CXRs that contained a range of pathologies. Jack-knife alternate free-response receiver operating characteristic (JAFROC) methodology was used to determine the performance of the observers (JAFROC v4.2). A noninferiority approach was used, with a predefined margin of clinical insignificance of 10% of average consultant radiologist diagnostic accuracy. The diagnostic accuracy of the reporting radiographers (figure of merit = 0.828, 95% confidence interval 0.808-0.847) was noninferior to the consultant radiologists (figure of merit = 0.788, 95% confidence interval 0.766-0.811), P < .0001. With appropriate postgraduate education, reporting radiographers are able to interpret CXRs at a level comparable to consultant radiologists.

One-view digital breast tomosynthesis as a stand-alone modality for breast cancer detection: do we need more?

PurposeTo compare the performance of one-view digital breast tomosynthesis (1v-DBT) to that of three other protocols combining DBT and mammography (DM) for breast cancer detection.Materials and methodsSix radiologists, three experienced with 1v-DBT in screening, retrospectively reviewed 181 cases (76 malignant, 50 benign, 55 normal) in two sessions. First, they scored sequentially: 1v-DBT (medio-lateral oblique, MLO), 1v-DBT (MLO) + 1v-DM (cranio-caudal, CC) and two-view DM + DBT (2v-DM+2v-DBT). The second session involved only 2v-DM. Lesions were scored using BI-RADS® and level of suspiciousness (1–10). Sensitivity, specificity, receiver operating characteristic (ROC) and jack-knife alternative free-response ROC (JAFROC) were computed.ResultsOn average, 1v-DBT was non-inferior to any of the other protocols in terms of JAFROC figure-of-merit, area under ROC curve, sensitivity or specificity (p>0.391). While readers inexperienced with 1v-DBT screening improved their sensitivity when adding more images (69–79 %, p=0.019), experienced readers showed similar sensitivity (76 %) and specificity (70 %) between 1v-DBT and 2v-DM+2v-DBT (p=0.482). Subanalysis by lesion type and breast density showed no difference among modalities.ConclusionDetection performance with 1v-DBT is not statistically inferior to 2v-DM or to 2v-DM+2v-DBT; its use as a stand-alone modality might be sufficient for readers experienced with this protocol.Key points• One-view breast tomosynthesis is not inferior to two-view digital mammography.• One-view DBT is not inferior to 2-view DM plus 2-view DBT.• Training may lead to 1v-DBT being sufficient for screening.