Jackknife Alternative Free-response Receiver Operating Characteristic Research Articles

Optimized signal of calcifications in wide-angle digital breast tomosynthesis: a virtual imaging trial.

Evaluate microcalcification detectability in digital breast tomosynthesis (DBT) and synthetic 2D mammography (SM) for different acquisition setups using a virtual imaging trial (VIT) approach. Medio-lateral oblique (MLO) DBT acquisitions on eight patients were performed at twice the automatic exposure controlled (AEC) dose. The noise was added to the projections to simulate a given dose trajectory. Virtual microcalcification models were added to a given projection set using an in-house VIT framework. Three setups were evaluated: (1) standard acquisition with 25 projections at AEC dose, (2) 25 projections with a convex dose distribution, and (3) sparse setup with 13 projections, every second one over the angular range. The total scan dose and angular range remained constant. DBT volume reconstruction and synthetic mammography image generation were performed using a Siemens prototype algorithm. Lesion detectability was assessed through a Jackknife-alternative free-response receiver operating characteristic (JAFROC) study with six observers. For DBT, the area under the curve (AUC) was 0.97 ± 0.01 for the standard, 0.95 ± 0.02 for the convex, and 0.89 ± 0.03 for the sparse setup. There was no significant difference between standard and convex dose distributions (p = 0.309). Sparse projections significantly reduced detectability (p = 0.001). Synthetic images had a higher AUC with the convex setup, though not significantly (p = 0.435). DBT required four times more reading time than synthetic mammography. A convex setup did not significantly improve detectability in DBT compared to the standard setup. Synthetic images exhibited a non-significant increase in detectability with the convex setup. Sparse setup significantly reduced detectability in both DBT and synthetic mammography. This virtual imaging trial study allowed the design and efficient testing of different dose distribution trajectories with real mammography images, using a dose-neutral protocol. • In DBT, a convex dose distribution did not increase the detectability of microcalcifications compared to the current standard setup but increased detectability for the SM images. • A sparse setup decreased microcalcification detectability in both DBT and SM images compared to the convex and current clinical setups. • Optimal microcalcification cluster detection in the system studied was achieved using either the standard or convex dose setting, with the default number of projections.

Development and multicenter validation of deep convolutional neural network-based detection of colorectal cancer on abdominal CT.

This study aims to develop computer-aided detection (CAD) for colorectal cancer (CRC) using abdominal CT based on a deep convolutional neural network. This retrospective study included consecutive patients with colorectal adenocarcinoma who underwent abdominal CT before CRC resection surgery (training set = 379, test set = 103). We customized the 3D U-Net of nnU-Net (CUNET) for CRC detection, which was trained with fivefold cross-validation using annotated CT images. CUNET was validated using datasets covering various clinical situations and institutions: an internal test set (n = 103), internal patients with CRC first determined by CT (n = 54) and asymptomatic CRC (n = 51), and an external validation set from two institutions (n = 60). During each validation, data from the healthy population were added (internal = 60; external = 130). CUNET was compared with other deep CNNs: residual U-Net and EfficientDet. The CAD performances were evaluated using per-CRC sensitivity (true positive/all CRCs), free-response receiver operating characteristic (FROC), and jackknife alternative FROC (JAFROC) curves. CUNET showed a higher maximum per-CRC sensitivity than residual U-Net and EfficientDet (internal test set 91.3% vs. 61.2%, and 64.1%). The per-CRC sensitivity of CUNET at false-positive rates of 3.0 was as follows: internal CRC determined by CT, 89.3%; internal asymptomatic CRC, 87.3%; and external validation, 89.6%. CUNET detected 69.2% (9/13) of CRCs missed by radiologists and 89.7% (252/281) of CRCs from all validation sets. CUNET can detect CRC on abdominal CT in patients with various clinical situations and from external institutions. • Customized 3D U-Net of nnU-Net (CUNET) can be applied to the opportunistic detection of colorectal cancer (CRC) in abdominal CT, helping radiologists detect unexpected CRC. • CUNET showed the best performance at false-positive rates ≥ 3.0, and 30.1% of false-positives were in the colorectum. CUNET detected 69.2% (9/13) of CRCs missed by radiologists and 87.3% (48/55) of asymptomatic CRCs. • CUNET detected CRCs in multiple validation sets composed of varying clinical situations and from different institutions, and CUNET detected 89.7% (252/281) of CRCs from all validation sets.

Detection of distant metastases and distant second primary cancers in head and neck squamous cell carcinoma: comparison of [18F]FDG PET/MRI and [18F]FDG PET/CT

PurposeThis prospective study aimed to compare the diagnostic performance of [18]FDG PET/MRI and PET/CT for the detection of distant metastases and distant second primary cancers in patients with head and neck squamous cell carcinoma (HNSCC).MethodsA total of 103 [18F]FDG PET/MRI examinations immediately followed by PET/CT were obtained in 82 consecutive patients for staging of primary HNSCC (n = 38), suspected loco-regional recurrence/follow-up (n = 41) or unknown primary HNSCC (n = 3). Histology and follow-up > 2 years formed the standard of reference. Blinded readers evaluated the anonymized PET/MRI and PET/CT examinations separately using a 5-point Likert score. Statistical analysis included: receiver operating characteristic (ROC) analysis, jackknife alternative free-response ROC (JAFROC) and region-of-interest (ROI)-based ROC to account for data clustering and sensitivity/specificity/accuracy comparisons for a score ≥ 3.ResultsDistant metastases and distant second primary cancers were present in 23/103 (22%) examinations in 16/82 (19.5%) patients, and they were more common in the post-treatment group (11/41, 27%) than in the primary HNSCC group (3/38, 8%), p = 0.039. The area under the curve (AUC) per patient/examination/lesion was 0.947 [0.927–1]/0.965 [0.917–1]/0.957 [0.928–0.987] for PET/MRI and 0.975 [0.950–1]/0.968 [0.920–1]/0.944 [0.910–0.979] for PET/CT, respectively (p > 0.05). The diagnostic performance of PET/MRI and PET/CT was similar according to JAFROC (p = 0.919) and ROI-based ROC analysis (p = 0.574). Sensitivity/specificity/accuracy for PET/MRI and PET/CT for a score ≥ 3 was 94%/88%/89% and 94%/91%/91% per patient, 96%/90%/91% and 96%/93%/93% per examination and 95%/85%/90% and 90%/86%/88% per lesion, respectively, p > 0.05.ConclusionsIn HNSCC patients, PET/MRI and PET/CT had a high and similar diagnostic performance for detecting distant metastases and distant second primary cancers.

Rethinking Annotation Granularity for Overcoming Shortcuts in Deep Learning-based Radiograph Diagnosis: A Multicenter Study.

To evaluate the ability of fine-grained annotations to overcome shortcut learning in deep learning (DL)-based diagnosis using chest radiographs. Two DL models were developed using radiograph-level annotations (disease present: yes or no) and fine-grained lesion-level annotations (lesion bounding boxes), respectively named CheXNet and CheXDet. A total of 34 501 chest radiographs obtained from January 2005 to September 2019 were retrospectively collected and annotated regarding cardiomegaly, pleural effusion, mass, nodule, pneumonia, pneumothorax, tuberculosis, fracture, and aortic calcification. The internal classification performance and lesion localization performance of the models were compared on a testing set (n = 2922); external classification performance was compared on National Institutes of Health (NIH) Google (n = 4376) and PadChest (n = 24 536) datasets; and external lesion localization performance was compared on the NIH ChestX-ray14 dataset (n = 880). The models were also compared with radiologist performance on a subset of the internal testing set (n = 496). Performance was evaluated using receiver operating characteristic (ROC) curve analysis. Given sufficient training data, both models performed similarly to radiologists. CheXDet achieved significant improvement for external classification, such as classifying fracture on NIH Google (CheXDet area under the ROC curve [AUC], 0.67; CheXNet AUC, 0.51; P < .001) and PadChest (CheXDet AUC, 0.78; CheXNet AUC, 0.55; P < .001). CheXDet achieved higher lesion detection performance than CheXNet for most abnormalities on all datasets, such as detecting pneumothorax on the internal set (CheXDet jackknife alternative free-response ROC [JAFROC] figure of merit [FOM], 0.87; CheXNet JAFROC FOM, 0.13; P < .001) and NIH ChestX-ray14 (CheXDet JAFROC FOM, 0.55; CheXNet JAFROC FOM, 0.04; P < .001). Fine-grained annotations overcame shortcut learning and enabled DL models to identify correct lesion patterns, improving the generalizability of the models.Keywords: Computer-aided Diagnosis, Conventional Radiography, Convolutional Neural Network (CNN), Deep Learning Algorithms, Machine Learning Algorithms, Localization Supplemental material is available for this article © RSNA, 2022.

Rib fracture detection in computed tomography images using deep convolutional neural networks.

To evaluate the rib fracture detection performance in computed tomography (CT) images using a software based on a deep convolutional neural network (DCNN) and compare it with the rib fracture diagnostic performance of doctors.We included CT images from 39 patients with thoracic injuries who underwent CT scans. In these images, 256 rib fractures were detected by two radiologists. This result was defined as the gold standard. The performances of rib fracture detection by the software and two interns were compared via the McNemar test and the jackknife alternative free-response receiver operating characteristic (JAFROC) analysis.The sensitivity of the DCNN software was significantly higher than those of both Intern A (0.645 vs 0.313; P < .001) and Intern B (0.645 vs 0.258; P < .001). Based on the JAFROC analysis, the differences in the figure-of-merits between the results obtained via the DCNN software and those by Interns A and B were 0.057 (95% confidence interval: −0.081, 0.195) and 0.071 (−0.082, 0.224), respectively. As the non-inferiority margin was set to −0.10, the DCNN software is non-inferior to the rib fracture detection performed by both interns.In the detection of rib fractures, detection by the DCNN software could be an alternative to the interpretation performed by doctors who do not have intensive training experience in image interpretation.

Added Value of Deep Learning-based Detection System for Multiple Major Findings on Chest Radiographs: A Randomized Crossover Study.

Background Previous studies assessing the effects of computer-aided detection on observer performance in the reading of chest radiographs used a sequential reading design that may have biased the results because of reading order or recall bias. Purpose To compare observer performance in detecting and localizing major abnormal findings including nodules, consolidation, interstitial opacity, pleural effusion, and pneumothorax on chest radiographs without versus with deep learning-based detection (DLD) system assistance in a randomized crossover design. Materials and Methods This study included retrospectively collected normal and abnormal chest radiographs between January 2016 and December 2017 (https://cris.nih.go.kr/; registration no. KCT0004147). The radiographs were randomized into two groups, and six observers, including thoracic radiologists, interpreted each radiograph without and with use of a commercially available DLD system by using a crossover design with a washout period. Jackknife alternative free-response receiver operating characteristic (JAFROC) figure of merit (FOM), area under the receiver operating characteristic curve (AUC), sensitivity, specificity, false-positive findings per image, and reading times of observers with and without the DLD system were compared by using McNemar and paired t tests. Results A total of 114 normal (mean patient age ± standard deviation, 51 years ± 11; 58 men) and 114 abnormal (mean patient age, 60 years ± 15; 75 men) chest radiographs were evaluated. The radiographs were randomized to two groups: group A (n = 114) and group B (n = 114). Use of the DLD system improved the observers' JAFROC FOM (from 0.90 to 0.95, P = .002), AUC (from 0.93 to 0.98, P = .002), per-lesion sensitivity (from 83% [822 of 990 lesions] to 89.1% [882 of 990 lesions], P = .009), per-image sensitivity (from 80% [548 of 684 radiographs] to 89% [608 of 684 radiographs], P = .009), and specificity (from 89.3% [611 of 684 radiographs] to 96.6% [661 of 684 radiographs], P = .01) and reduced the reading time (from 10-65 seconds to 6-27 seconds, P < .001). The DLD system alone outperformed the pooled observers (JAFROC FOM: 0.96 vs 0.90, respectively, P = .007; AUC: 0.98 vs 0.93, P = .003). Conclusion Observers including thoracic radiologists showed improved performance in the detection and localization of major abnormal findings on chest radiographs and reduced reading time with use of a deep learning-based detection system. © RSNA, 2021 Online supplemental material is available for this article.

Impact of Hours Awake and Hours Slept at Night on Radiologists' Mammogram Interpretations.

To examine whether radiologists' mammogram reading performance varies according to how long they have been awake ("hours awake") and the number of hours they slept ("hours slept") the night before a reading session. Retrospective data were retrieved from the BreastScreen Reader Assessment Strategy database. Malignancy-enriched mammographic readings were performed by 133 radiologists. Information on their hours awake and hours slept was collected. Analysis of covariance was performed to determine whether these two variables influenced radiologists' sensitivity, specificity, lesion sensitivity, receiver operating characteristic (ROC) curve, and jackknife alternative free-response ROC. Radiologists were divided into a more experienced and a less experienced groups (based on reading ≥2,000 and <2,000 mammogram readings per year, respectively). The hours awake significantly influenced less experienced radiologists' lesion sensitivity (F6,63= 2.51; P= .03). Those awake for <2 hours had significantly lower lesion sensitivity than those awake for 8 to 10 hours (P= .01), and those awake for 4 to 6 hours had significantly lower lesion sensitivity than those awake for 8 to 10 hours (P= .002) and 10 to 12 hours (P= .02). The hours slept also influenced the ROC values of less experienced radiologists (F1,68= 4.96; P= .02). Radiologists with up to 6 hours of sleep had a significantly lower value (0.72) than those who had slept more than 6 hours (0.77). No statistically significant findings were noted for more experienced radiologists. Inexperienced radiologists' performance may be influenced by the hours awake and hours slept before reading sessions.

Impact of time of day on radiology image interpretations

To examine the impact of the time of day on radiologists' mammography reading performance. Retrospective mammographic reading assessment data were collected from the BreastScreen Reader Assessment Strategy database and included timestamps of the readings and reader-specific demographic data of 197 radiologists. The radiologists performed the readings in a workshop setting with test case sets enriched with malignancies (one-third of cases were malignant). The collected data were evaluated with an analysis of covariance to determine whether time of day influenced radiologists' specificity, lesion sensitivity or the jackknife alternative free-response receiver operating characteristic (JAFROC). After adjusting for radiologist experience and fellowship, specificity varied significantly by time of day (p=0.027), but there was no evidence of any significant impact on lesion sensitivity (p=0.441) or JAFROC (p=0.120). The collected data demonstrated that specificity during the late morning (10.00-12.00) was 71.7%; this was significantly lower than in the early morning (08.00-10.00) and mid-afternoon (14.00-16.00), which were 82.74% (p=0.003) and 81.39% (p=0.031), respectively. Specificity during the late afternoon (16.00-18.00) was 73.95%; this was significantly lower than in the early morning (08.00-10.00) and mid-afternoon (14.00-16.00), which were 82.74% (p=0.003) and 81.39% (p=0.031), respectively. The results indicated that the time of day may influence radiologists' performance, specifically their ability to identify normal images correctly.

Development and Validation of Deep Learning-based Automatic Detection Algorithm for Malignant Pulmonary Nodules on Chest Radiographs.

Purpose To develop and validate a deep learning-based automatic detection algorithm (DLAD) for malignant pulmonary nodules on chest radiographs and to compare its performance with physicians including thoracic radiologists. Materials and Methods For this retrospective study, DLAD was developed by using 43 292 chest radiographs (normal radiograph-to-nodule radiograph ratio, 34 067:9225) in 34 676 patients (healthy-to-nodule ratio, 30 784:3892; 19 230 men [mean age, 52.8 years; age range, 18-99 years]; 15 446 women [mean age, 52.3 years; age range, 18-98 years]) obtained between 2010 and 2015, which were labeled and partially annotated by 13 board-certified radiologists, in a convolutional neural network. Radiograph classification and nodule detection performances of DLAD were validated by using one internal and four external data sets from three South Korean hospitals and one U.S. hospital. For internal and external validation, radiograph classification and nodule detection performances of DLAD were evaluated by using the area under the receiver operating characteristic curve (AUROC) and jackknife alternative free-response receiver-operating characteristic (JAFROC) figure of merit (FOM), respectively. An observer performance test involving 18 physicians, including nine board-certified radiologists, was conducted by using one of the four external validation data sets. Performances of DLAD, physicians, and physicians assisted with DLAD were evaluated and compared. Results According to one internal and four external validation data sets, radiograph classification and nodule detection performances of DLAD were a range of 0.92-0.99 (AUROC) and 0.831-0.924 (JAFROC FOM), respectively. DLAD showed a higher AUROC and JAFROC FOM at the observer performance test than 17 of 18 and 15 of 18 physicians, respectively (P < .05), and all physicians showed improved nodule detection performances with DLAD (mean JAFROC FOM improvement, 0.043; range, 0.006-0.190; P < .05). Conclusion This deep learning-based automatic detection algorithm outperformed physicians in radiograph classification and nodule detection performance for malignant pulmonary nodules on chest radiographs, and it enhanced physicians' performances when used as a second reader. © RSNA, 2018 Online supplemental material is available for this article.

Observer Performance with Varying Radiation Dose and Reconstruction Methods for Detection of Hepatic Metastases.

Purpose To estimate the ability of lower dose levels and iterative reconstruction (IR) to display hepatic metastases that can be detected by radiologists. Materials and Methods Projection data from 83 contrast agent-enhanced CT examinations were collected. Metastases were defined by histopathologic analysis or progression and regression. Lower radiation dose configurations were reconstructed at five dose levels with filtered back projection (FBP) and IR (automatic exposure control settings: 80, 100, 120, 160, and 200 quality reference mAs [QRM]). Three abdominal radiologists circumscribed metastases, indicating confidence (confidence range, 0-100) and image quality. Noninferiority was assessed by using jackknife alternative free-response receiver operating characteristic (JAFROC) analysis (noninferiority limit, -0.10) and reader agreement rules, which required identification of metastases identified at routine dose, and no nonlesion localizations in patients negative for metastases, in 71 or more patient CT examinations (of 83), for each configuration. Results There were 123 hepatic metastases (mean size, 1.4 cm; median volume CT dose index and size-specific dose estimate, 11.0 and 13.4 mGy, respectively). By using JAFROC figure of merit, 100 QRM FBP did not meet noninferiority criteria and had estimated performance difference from routine dose of -0.08 (95% confidence interval: -0.11, -0.04). Preset reader agreement rules were not met for 100 QRM IR or 80 QRM IR, but were met for doses 120 QRM or higher (ie, size-specific dose estimate ≥ 8.0 mGy). IR improved image quality (P < .05) but not reader performance. Other than 160 QRM IR, lower dose levels were associated with reduced confidence in metastasis detection (P < .001). Conclusion For detection of hepatic metastases by using contrast-enhanced CT, dose levels that corresponded to 120 quality reference mAs (size-specific dose estimate, 8.0 mGy) and higher performed similarly to 200 quality reference mAs with filtered back projection. © RSNA, 2018 Online supplemental material is available for this article.

Reduction of the radiation dose and the amount of contrast material in hepatic dynamic CT using low tube voltage and adaptive iterative dose reduction 3-dimensional.

The purpose of this study was to prospectively evaluate the image quality and the diagnostic ability of low tube voltage and reduced contrast material dose hepatic dynamic computed tomography (CT) reconstructed with adaptive iterative dose reduction 3-dimensional (AIDR 3D).Eighty-nine patients underwent hepatic dynamic CT using one of the 2 protocols: tube voltage of 120 kVp, contrast dose of 600 mgI/kg, and filtered back projection in Protocol A (n = 46), and tube voltage of 100 kVp, contrast dose of 500 mgI/kg, and AIDR 3D in Protocol B (n = 43). The volume CT dose index (CTDIvol) and size-specific dose estimates (SSDEs) were compared between the 2 groups. Objective image noise and tumor to liver contrast-to-noise ratio (CNR) were also compared. Three radiologists independently reviewed image quality. The jackknife alternative free-response receiver-operating characteristic (JAFROC) analysis was performed to compare diagnostic performance.The mean CTDIvol and SSDE of Protocol B (14.3 and 20.2, respectively) were significantly lower than those of Protocol A (22.1 and 31.4, P < .001). There were no significant differences in either objective image noise or CNR. In the qualitative analysis, 2 readers assigned significant lower scores to images of Protocol B for at least one of the 3 phases regarding overall image quality (P < .05). There was no significant difference in the JAFROC1 figure of merit between protocols.Low tube voltage CT with AIDR 3D yielded a reduction in radiation dose and in the amount of contrast material while maintaining diagnostic performance.

Triple Arterial Phase CT of the Liver with Radiation Dose Equivalent to That of Single Arterial Phase CT: Initial Experience.

Purpose To develop and evaluate a triple arterial phase CT liver protocol with a similar radiation dose to that of standard single arterial phase CT in study subjects suspected of having hepatocellular carcinoma (HCC). Materials and Methods The study consisted of a retrospective part A for protocol development (n = 15) and a prospective part B to evaluate diagnostic accuracy (n = 38). All 53 participants underwent perfusion CT with 50 mL contrast material between August 2013 and September 2014. Group B underwent an additional standard multiphasic liver CT examination with 120 mL of contrast material (range, 70-143 mL). Image sets from triple arterial phase imaging were reconstructed from perfusion CT by fusing images from three dedicated arterial time points. Triple arterial phase CT and standard single arterial phase CT were compared by two readers, who assessed subjective image quality and HCC detection rate. A third reader served as reference reader and assessed objective image quality. The paired Student t test, Wilcoxon signed rank test, jackknife alternative free-response receiver operating characteristic (JAFROC), and JAFROC curve were applied. Results The mean volume CT dose index was 11.6 mGy for triple arterial phase CT and 11.9 mGy for standard single arterial phase CT (P = .73). Triple arterial phase CT showed lower image noise and better contrast-to-noise ratio compared with standard single arterial phase CT (P < .001 and P = .032, respectively); however, there was no significant difference in lesion-to-liver-contrast ratio (P = .31). Subjective image quality was good for both protocols. The detection rate of the 65 HCC lesions was 82% for reader 1 and 83% for reader 2 at triple arterial phase CT and 80% for reader 1 and 77% for reader 2 at standard single arterial phase CT (P = .4). Conclusion Triple arterial phase imaging is feasible at the same radiation dose as that used for standard single arterial phase CT. Triple arterial phase imaging provides equivalent to superior image quality and equal HCC detection rate despite the use of less than half the contrast material dose used at standard single arterial phase CT. © RSNA, 2018.

Assessment of Jordanian Radiologist Performance in the Detection of Breast Cancers

This study aims to monitor diagnostic accuracy amongst Jordanian mammography readers and identify parameters linked to higher levels of performance. In this study, we have used the Breast Screen Reader Assessment Strategy (BREAST) platform to facilitate 27 radiologists in reading a case set of 60 digital mammograms, 20 of which included cancers. Each case consisted of the four standard cranio-caudal (CC) and medio-lateral oblique (MLO) projections. All radiologists were registered to read mammograms at their workplace by the Jordanian Ministry of Health. Each reader was asked to locate any malignancies, provide a confidence rating using a scale of 1 - 5, and identify the type of appearance. All images were displayed using 8 MP monitor, supported by radiology workstations with full image manipulation facilities. Results were evaluated using Jackknife Alternative Free-Response Receiver Operating Characteristic (JAFROC). Demographics obtained from each radiologist regarding their experience, qualifications and breast-reading activities were correlated against JAFROC scores using Spearman techniques. The results showed that the mean JAFROC score was 0. 52 (95% confidence interval (CI): 0.46, 0.58); location sensitivity score was 0. 41 (95% CI: 0.41, 0.56); specificity score was 0.73 (95% CI: 0.68, 0.83). Higher performance in term of JAFROC scores was directly related to number of years since professional qualification (r = 0.433; p = 0.024), number of years reading breast images (r = 0.62; r = 0.001) and number of mammography images read per year (r = 0.69; p = 0.001). On the other hand, higher performance was inversely linked to the frequency of reading other modalities per week (r = -0.48; p = 0.010). No other statistical differences were significant. Finally, higher radiologists’ performance in cancer detection is correlated with increasing the number of mammograms reads per week.

One-view digital breast tomosynthesis as a stand-alone modality for breast cancer detection: do we need more?

PurposeTo compare the performance of one-view digital breast tomosynthesis (1v-DBT) to that of three other protocols combining DBT and mammography (DM) for breast cancer detection.Materials and methodsSix radiologists, three experienced with 1v-DBT in screening, retrospectively reviewed 181 cases (76 malignant, 50 benign, 55 normal) in two sessions. First, they scored sequentially: 1v-DBT (medio-lateral oblique, MLO), 1v-DBT (MLO) + 1v-DM (cranio-caudal, CC) and two-view DM + DBT (2v-DM+2v-DBT). The second session involved only 2v-DM. Lesions were scored using BI-RADS® and level of suspiciousness (1–10). Sensitivity, specificity, receiver operating characteristic (ROC) and jack-knife alternative free-response ROC (JAFROC) were computed.ResultsOn average, 1v-DBT was non-inferior to any of the other protocols in terms of JAFROC figure-of-merit, area under ROC curve, sensitivity or specificity (p>0.391). While readers inexperienced with 1v-DBT screening improved their sensitivity when adding more images (69–79 %, p=0.019), experienced readers showed similar sensitivity (76 %) and specificity (70 %) between 1v-DBT and 2v-DM+2v-DBT (p=0.482). Subanalysis by lesion type and breast density showed no difference among modalities.ConclusionDetection performance with 1v-DBT is not statistically inferior to 2v-DM or to 2v-DM+2v-DBT; its use as a stand-alone modality might be sufficient for readers experienced with this protocol.Key points• One-view breast tomosynthesis is not inferior to two-view digital mammography.• One-view DBT is not inferior to 2-view DM plus 2-view DBT.• Training may lead to 1v-DBT being sufficient for screening.

Low-dose CT for the detection and classification of metastatic liver lesions: Results of the 2016 Low Dose CT Grand Challenge.

Using common datasets, to estimate and compare the diagnostic performance of image-based denoising techniques or iterative reconstruction algorithms for the task of detecting hepatic metastases. Datasets from contrast-enhanced CT scans of the liver were provided to participants in an NIH-, AAPM- and Mayo Clinic-sponsored Low Dose CT Grand Challenge. Training data included full-dose and quarter-dose scans of the ACR CT accreditation phantom and 10 patient examinations; both images and projections were provided in the training data. Projection data were supplied in a vendor-neutral standardized format (DICOM-CT-PD). Twenty quarter-dose patient datasets were provided to each participant for testing the performance of their technique. Images were provided to sites intending to perform denoising in the image domain. Fully preprocessed projection data and statistical noise maps were provided to sites intending to perform iterative reconstruction. Upon return of the denoised or iteratively reconstructed quarter-dose images, randomized, blinded evaluation of the cases was performed using a Latin Square study design by 11 senior radiology residents or fellows, who marked the locations of identified hepatic metastases. Markings were scored against reference locations of clinically or pathologically demonstrated metastases to determine a per-lesion normalized score and a per-case normalized score (a faculty abdominal radiologist established the reference location using clinical and pathological information). Scores increased for correct detections; scores decreased for missed or incorrect detections. The winner for the competition was the entry that produced the highest total score (mean of the per-lesion and per-case normalized score). Reader confidence was used to compute a Jackknife alternative free-response receiver operating characteristic (JAFROC) figure of merit, which was used for breaking ties. 103 participants from 90 sites and 26 countries registered to participate. Training data were shared with 77 sites that completed the data sharing agreements. Subsequently, 41 sites downloaded the 20 test cases, which included only the 25% dose data (CTDIvol=3.0±1.8mGy, SSDE=3.5±1.3mGy). 22 sites submitted results for evaluation. One site provided binary images and one site provided images with severe artifacts; cases from these sites were excluded from review and the participants removed from the challenge. The mean (range) per-lesion and per-case normalized scores were -24.2% (-75.8%, 3%) and 47% (10%, 70%), respectively. Compared to reader results for commercially reconstructed quarter-dose images with no noise reduction, 11 of the 20 sites showed a numeric improvement in the mean JAFROC figure of merit. Notably two sites performed comparably to the reader results for full-dose commercial images. The study was not designed for these comparisons, so wide confidence intervals surrounded these figures of merit and the results should be used only to motivate future testing. Infrastructure and methodology were developed to rapidly estimate observer performance for liver metastasis detection in low-dose CT examinations of the liver after either image-based denoising or iterative reconstruction. The results demonstrated large differences in detection and classification performance between noise reduction methods, although the majority of methods provided some improvement in performance relative to the commercial quarter-dose images with no noise reduction applied.

Diagnostic accuracy of gadoxetic acid-enhanced MR for small hypervascular hepatocellular carcinoma and the concordance rate of Liver Imaging Reporting and Data System (LI-RADS).

Background & aimsTo assess diagnostic accuracy of gadoxetic acid–enhanced MR for small hypervascular hepatocellular carcinoma (HCC) detected by C-arm CT and concordance rate of Liver Imaging Reporting and Data System (LI-RADS).MethodsIn this retrospective study, we recruited 4,544 patients suspected of having HCC underwent C-arm CT from November 2008 to May 2013. Among these patients, gadoxetic acid–enhanced MR was performed in 167 patients with HCC (n = 379; 257 > 1 cm, 122 ≤ 1 cm). HCC was confirmed by MR, CT, or follow-up images. Two radiologists graded likelihood of HCC and assessed MR features. Jackknife alternative free-response receiver operating characteristic (JAFROC) analysis was performed. All HCCs were evaluated concordance rate of LI-RADS.ResultsMean JAFROC figure of merit for large (>1-cm) HCC was 0.948, while that for small HCC was 0.787 with fair agreement (κ = 0.409). Mean sensitivity and positive predictive value (PPV) were 91% and 90% for large HCC versus 63.0% and 79% for small HCC, respectively. Seventeen of 122 small HCCs (13.9%) were not visible on MR. Among 379 HCCs, 99 met LR-5, and 259 met LR-4. Common features for small HCC included arterial enhancement (81.9%), hepatobiliary phase hypointensity (80.3%), and delayed washout (72.9%).ConclusionDiagnostic accuracy of gadoxetic acid–enhanced MR imaging for small, hypervascular HCCs (Mean figure of merit = 0.787) was still low compared with large HCC (Mean figure of merit = 0.948). LR-5 and LR-4 covered 94% (358/379) of the HCCs.

Image processing can cause some malignant soft-tissue lesions to be missed in digital mammography images

To investigate the effect of image processing on cancer detection in mammography. An observer study was performed using 349 digital mammography images of women with normal breasts, calcification clusters, or soft-tissue lesions including 191 subtle cancers. Images underwent two types of processing: FlavourA (standard) and FlavourB (added enhancement). Six observers located features in the breast they suspected to be cancerous (4,188 observations). Data were analysed using jackknife alternative free-response receiver operating characteristic (JAFROC) analysis. Characteristics of the cancers detected with each image processing type were investigated. For calcifications, the JAFROC figure of merit (FOM) was equal to 0.86 for both types of image processing. For soft-tissue lesions, the JAFROC FOM were better for FlavourA (0.81) than FlavourB (0.78); this difference was significant (p=0.001). Using FlavourA a greater number of cancers of all grades and sizes were detected than with FlavourB. FlavourA improved soft-tissue lesion detection in denser breasts (p=0.04 when volumetric density was over 7.5%) CONCLUSIONS: The detection of malignant soft-tissue lesions (which were primarily invasive) was significantly better with FlavourA than FlavourB image processing. This is despite FlavourB having a higher contrast appearance often preferred by radiologists. It is important that clinical choice of image processing is based on objective measures.

Cascade convolutional neural networks for automatic detection of thyroid nodules in ultrasound images.

It is very important for calculation of clinical indices and diagnosis to detect thyroid nodules from ultrasound images. However, this task is a challenge mainly due to heterogeneous thyroid nodules with distinct components are similar to background in ultrasound images. In this study, we employ cascade deep convolutional neural networks (CNNs) to develop and evaluate a fully automatic detection of thyroid nodules from 2D ultrasound images. Our cascade CNNs are a type of hybrid model, consisting of two different CNNs and a new splitting method. Specifically, it employs a deep CNN to learn the segmentation probability maps from the ground true data. Then, all the segmentation probability maps are split into different connected regions by the splitting method. Finally, another deep CNN is used to automatically detect the thyroid nodules from ultrasound thyroid images. Experiment results illustrate the cascade CNNs are very effective in detection of thyroid nodules. Specially, the value of area under the curve of receiver operating characteristic is 98.51%. The Free-response receiver operating characteristic (FROC) and jackknife alternative FROC (JAFROC) analyses show a significant improvement in the performance of our cascade CNNs compared to that of other methods. The multi-view strategy can improve the performance of cascade CNNs. Moreover, our special splitting method can effectively separate different connected regions so that the second CNN can correctively gain the positive and negative samples according to the automatic labels. The experiment results demonstrate the potential clinical applications of this proposed method. This technique can offer physicians an objective second opinion, and reduce their heavy workload so as to avoid misdiagnosis causes because of excessive fatigue. In addition, it is easy and reproducible for a person without medical expertise to diagnose thyroid nodules.

A JAFROC study of nodule detection performance in CT images of a thorax acquired during PET/CT

PurposeTwo types of CT images (modalities) are acquired in PET/CT: for attenuation correction (AC) and diagnosis. The purpose of the study was to compare nodule detection and localization performance between these two modalities. MethodsCT images, using both modalities, of an anthropomorphic chest phantom containing zero or more simulated spherical nodules of 5, 8, 10 and 12 mm diameters and contrasts −800, −630 and 100 HU were acquired. An observer performance study using nine observers interpreting 45 normal (zero nodules) images and 47 abnormal images (1–3 nodules; average 1.26) was conducted using the free-response receiver operating characteristic (FROC) paradigm. Data were analysed using an R software package implemented jackknife alternative FROC (JAFROC) analysis. Both empirical areas under the equally weighted AFROC curve (wAFROC) and under the highest rating inferred ROC (HR-ROC) curve were used as figures of merit (FOM). To control the probability of Type I error test alpha was set at 0.05. ResultsNodule detection as measured by either FOM was significantly better on the diagnostic quality images (2nd modality), irrespective of the method of analysis, [reader averaged inter-modality wAFROC FOM difference = −0.07 (−0.11,−0.04); reader averaged inter-modality HR-ROC FOM difference = −0.05 (−0.09, −0.01)]. ConclusionNodule detection was statistically worse on images acquired for AC; suggesting that images acquired for AC should not be used to evaluate pulmonary pathology.

“Pine tree” appearance on 18F-FDG PET/CT MIP image in spinal tuberculosis

Two types of CT images (modalities) are acquired in PET/CT: for attenuation correction (AC) and diagnosis. The purpose of the study was to compare nodule detection and localization performance between these two modalities.CT images, using both modalities, of an anthropomorphic chest phantom containing zero or more simulated spherical nodules of 5, 8, 10 and 12 mm diameters and contrasts −800, −630 and 100 HU were acquired. An observer performance study using nine observers interpreting 45 normal (zero nodules) images and 47 abnormal images (1–3 nodules; average 1.26) was conducted using the free-response receiver operating characteristic (FROC) paradigm. Data were analysed using an R software package implemented jackknife alternative FROC (JAFROC) analysis. Both empirical areas under the equally weighted AFROC curve (wAFROC) and under the highest rating inferred ROC (HR-ROC) curve were used as figures of merit (FOM). To control the probability of Type I error test alpha was set at 0.05.Nodule detection as measured by either FOM was significantly better on the diagnostic quality images (2nd modality), irrespective of the method of analysis, [reader averaged inter-modality wAFROC FOM difference = −0.07 (−0.11,−0.04); reader averaged inter-modality HR-ROC FOM difference = −0.05 (−0.09, −0.01)].Nodule detection was statistically worse on images acquired for AC; suggesting that images acquired for AC should not be used to evaluate pulmonary pathology.