Free-response Receiver Operating Characteristic Curve Research Articles

Simultaneous detection of dental caries and fissure sealant in intraoral photos by deep learning: a pilot study

BackgroundDeep learning, as an artificial intelligence method has been proved to be powerful in analyzing images. The purpose of this study is to construct a deep learning-based model (ToothNet) for the simultaneous detection of dental caries and fissure sealants in intraoral photos.MethodsA total of 1020 intraoral photos were collected from 762 volunteers. Teeth, caries and sealants were annotated by two endodontists using the LabelMe tool. ToothNet was developed by modifying the YOLOX framework for simultaneous detection of caries and fissure sealants. The area under curve (AUC) in the receiver operating characteristic curve (ROC) and free-response ROC (FROC) curves were used to evaluate model performance in the following aspects: (i) classification accuracy of detecting dental caries and fissure sealants from a photograph (image-level); and (ii) localization accuracy of the locations of predicted dental caries and fissure sealants (tooth-level). The performance of ToothNet and dentist with 1year of experience (1-year dentist) were compared at tooth-level and image-level using Wilcoxon test and DeLong test.ResultsAt the image level, ToothNet achieved an AUC of 0.925 (95% CI, 0.880–0.958) for caries detection and 0.902 (95% CI, 0.853–0.940) for sealant detection. At the tooth level, with a confidence threshold of 0.5, the sensitivity, precision, and F1-score for caries detection were 0.807, 0.814, and 0.810, respectively. For fissure sealant detection, the values were 0.714, 0.750, and 0.731. Compared with ToothNet, the 1-year dentist had a lower F1 value (0.599, p < 0.0001) and AUC (0.749, p < 0.0001) in caries detection, and a lower F1 value (0.727, p = 0.023) and similar AUC (0.829, p = 0.154) in sealant detection.ConclusionsThe proposed deep learning model achieved multi-task simultaneous detection in intraoral photos and showed good performance in the detection of dental caries and fissure sealants. Compared with 1-year dentist, the model has advantages in caries detection and is equivalent in fissure sealants detection.

Development and multicenter validation of deep convolutional neural network-based detection of colorectal cancer on abdominal CT.

This study aims to develop computer-aided detection (CAD) for colorectal cancer (CRC) using abdominal CT based on a deep convolutional neural network. This retrospective study included consecutive patients with colorectal adenocarcinoma who underwent abdominal CT before CRC resection surgery (training set = 379, test set = 103). We customized the 3D U-Net of nnU-Net (CUNET) for CRC detection, which was trained with fivefold cross-validation using annotated CT images. CUNET was validated using datasets covering various clinical situations and institutions: an internal test set (n = 103), internal patients with CRC first determined by CT (n = 54) and asymptomatic CRC (n = 51), and an external validation set from two institutions (n = 60). During each validation, data from the healthy population were added (internal = 60; external = 130). CUNET was compared with other deep CNNs: residual U-Net and EfficientDet. The CAD performances were evaluated using per-CRC sensitivity (true positive/all CRCs), free-response receiver operating characteristic (FROC), and jackknife alternative FROC (JAFROC) curves. CUNET showed a higher maximum per-CRC sensitivity than residual U-Net and EfficientDet (internal test set 91.3% vs. 61.2%, and 64.1%). The per-CRC sensitivity of CUNET at false-positive rates of 3.0 was as follows: internal CRC determined by CT, 89.3%; internal asymptomatic CRC, 87.3%; and external validation, 89.6%. CUNET detected 69.2% (9/13) of CRCs missed by radiologists and 89.7% (252/281) of CRCs from all validation sets. CUNET can detect CRC on abdominal CT in patients with various clinical situations and from external institutions. • Customized 3D U-Net of nnU-Net (CUNET) can be applied to the opportunistic detection of colorectal cancer (CRC) in abdominal CT, helping radiologists detect unexpected CRC. • CUNET showed the best performance at false-positive rates ≥ 3.0, and 30.1% of false-positives were in the colorectum. CUNET detected 69.2% (9/13) of CRCs missed by radiologists and 87.3% (48/55) of asymptomatic CRCs. • CUNET detected CRCs in multiple validation sets composed of varying clinical situations and from different institutions, and CUNET detected 89.7% (252/281) of CRCs from all validation sets.

Assessing clinical efficacy of polyp detection models using open-access datasets.

Ensuring accurate polyp detection during colonoscopy is essential for preventing colorectal cancer (CRC). Recent advances in deep learning-based computer-aided detection (CADe) systems have shown promise in enhancing endoscopists' performances. Effective CADe systems must achieve high polyp detection rates from the initial seconds of polyp appearance while maintaining low false positive (FP) detection rates throughout the procedure. We integrated four open-access datasets into a unified platform containing over 340,000 images from various centers, including 380 annotated polyps, with distinct data splits for comprehensive model development and benchmarking. The REAL-Colon dataset, comprising 60 full-procedure colonoscopy videos from six centers, is used as the fifth dataset of the platform to simulate clinical conditions for model evaluation on unseen center data. Performance assessment includes traditional object detection metrics and new metrics that better meet clinical needs. Specifically, by defining detection events as sequences of consecutive detections, we compute per-polyp recall at early detection stages and average per-patient FPs, enabling the generation of Free-Response Receiver Operating Characteristic (FROC) curves. Using YOLOv7, we trained and tested several models across the proposed data splits, showcasing the robustness of our open-access platform for CADe system development and benchmarking. The introduction of new metrics allows for the optimization of CADe operational parameters based on clinically relevant criteria, such as per-patient FPs and early polyp detection. Our findings also reveal that omitting full-procedure videos leads to non-realistic assessments and that detecting small polyp bounding boxes poses the greatest challenge. This study demonstrates how newly available open-access data supports ongoing research progress in environments that closely mimic clinical settings. The introduced metrics and FROC curves illustrate CADe clinical efficacy and can aid in tuning CADe hyperparameters.

Fully-automated detection of small bowel carcinoid tumors in CT scans using deep learning.

Small bowel carcinoid tumor is a rare neoplasm and increasing in incidence. Patients with small bowel carcinoid tumors often experience long delays in diagnosis due to the vague symptoms, slow growth of tumors, and lack of clinician awareness. Computed tomography (CT) is the most common imaging study for diagnosis of small bowel carcinoid tumor. It is often used with positron emission tomography (PET) to capture anatomical and functional aspects of carcinoid tumors and thus to increase thesensitivity. We compared three different kinds of methods for the automatic detection of small bowel carcinoid tumors on CT scans, which is the first to the best of ourknowledge. Thirty-three preoperative CT scans of 33 unique patients with surgically-proven carcinoid tumors within the small bowel were collected. Ground-truth segmentation of tumors was drawn on CT scans by referring to available 18 F-DOPA PET scans and the corresponding radiology report. These scans were split into the trainval set (n = 24) and the test positive set (n= 9). Additionally, 22 CT scans of 22 unique patients who had no evidence of the tumor were collected to comprise the test negative set. We compared three different kinds of detection methods, which are detection network, patch-based classification, and segmentation-based methods. We also investigated the usefulness of small bowel segmentation for reduction of false positives (FPs) for each method. Free-response receiver operating characteristic (FROC) curves and receiver operating characteristic (ROC) curves were used for lesion- and patient-level evaluations, respectively. Statistical analyses comparing the FROC and ROC curves were alsoperformed. The detection network method performed the best among the compared methods. For lesion-level detection, the detection network method, without the small bowel segmentation-based filtering, achieved sensitivity values of (60.8%, 81.1%, 82.4%, 86.5%) at per-scan FP rates of (1, 2, 4 ,8), respectively. The use of the small bowel segmentation did not improve the performance ( ). For patient-level detection, again the detection network method, but with the small bowel segmentation-based filtering, achieved the highest AUC of 0.86 with a sensitivity of 78% and specificity of 82% at the Youdenpoint. The carcinoid tumors in this patient population were very small and potentially difficult to diagnose. The presented method showed reasonable sensitivity at small numbers of FPs for lesion-level detection. It also achieved a promising AUC for patient-level detection. The method may have clinical application in patients with this rare and difficult to detectdisease.

A Deep Learning Framework for the Detection and Quantification of Reticular Pseudodrusen and Drusen on Optical Coherence Tomography.

The purpose of this study was to develop and validate a deep learning (DL) framework for the detection and quantification of reticular pseudodrusen (RPD) and drusen on optical coherence tomography (OCT) scans. A DL framework was developed consisting of a classification model and an out-of-distribution (OOD) detection model for the identification of ungradable scans; a classification model to identify scans with drusen or RPD; and an image segmentation model to independently segment lesions as RPD or drusen. Data were obtained from 1284 participants in the UK Biobank (UKBB) with a self-reported diagnosis of age-related macular degeneration (AMD) and 250 UKBB controls. Drusen and RPD were manually delineated by five retina specialists. The main outcome measures were sensitivity, specificity, area under the receiver operating characteristic (ROC) curve (AUC), kappa, accuracy, intraclass correlation coefficient (ICC), and free-response receiver operating characteristic (FROC) curves. The classification models performed strongly at their respective tasks (0.95, 0.93, and 0.99 AUC, respectively, for the ungradable scans classifier, the OOD model, and the drusen and RPD classification models). The mean ICC for the drusen and RPD area versus graders was 0.74 and 0.61, respectively, compared with 0.69 and 0.68 for intergrader agreement. FROC curves showed that the model's sensitivity was close to human performance. The models achieved high classification and segmentation performance, similar to human performance. Application of this robust framework will further our understanding of RPD as a separate entity from drusen in both research and clinical settings.

The Use of Deep Learning-Based Computer Diagnostic Algorithm for Detection of Lymph Node Metastases of Gastric Adenocarcinoma.

Objectives. The diversifying modalities of treatment for gastric cancer raise urgent demands for the rapid and precise diagnosis of metastases in regional lymph nodes, thereby significantly impact the workload of pathologists. Meanwhile, the recent advent of whole-slide scanners and deep-learning techniques have enabled the computer-assisted analysis of histopathological images, which could help to alleviate this impact. Thus, we developed a deep learning-based diagnostic algorithm to detect lymph node metastases of gastric adenocarcinoma and evaluated its performance. Methods. We randomly selected 20 patients with gastric adenocarcinoma who underwent surgery as definitive treatment and were found to be node metastasis-positive. HEMATOXYLIN-eosin (HE) stained glass slides, including a total of 51 metastasis-positive nodes, were retrieved from the specimens of these cases. Other slides with 776 metastasis-negative nodes were also retrieved from other twenty cases with the same disease that were diagnosed as metastasis-negative by the final pathological examinations. All glass slides were digitized using a whole-slide scanner. A deep-learning algorithm to detect metastases was developed using the data in which metastasis-positive parts of the images were annotated by a well-trained pathologist, and its performance in detecting metastases was evaluated. Results. Cross-validation analysis indicated an area of 0.9994 under the receiver operating characteristic curve. Free-response receiver operating characteristic curve (FROC) analysis indicated a sensitivity of 1.00 with three false positives. Further evaluation using an independent dataset also showed similar level of accuracies. Conclusion. This deep learning-based diagnosis-aid system is a promising tool that can assist pathologists involved in gastric cancer care and reduce their workload.

Deep Learning Empowers Lung Cancer Screening Based on Mobile Low-Dose Computed Tomography in Resource-Constrained Sites.

Existing challenges of lung cancer screening included non-accessibility of computed tomography (CT) scanners and inter-reader variability, especially in resource-limited areas. The combination of mobile CT and deep learning technique has inspired innovations in the routine clinical practice. This study recruited participants prospectively in two rural sites of western China. A deep learning system was developed to assist clinicians to identify the nodules and evaluate the malignancy with state-of-the-art performance assessed by recall, free-response receiver operating characteristic curve (FROC), accuracy (ACC), area under the receiver operating characteristic curve (AUC). This study enrolled 12,360 participants scanned by mobile CT vehicle, and detected 9511 (76.95%) patients with pulmonary nodules. Majority of participants were female (8169, 66.09%), and never-smokers (9784, 79.16%). After 1-year follow-up, 86 patients were diagnosed with lung cancer, with 80 (93.03%) of adenocarcinoma, and 73 (84.88%) at stage I. This deep learning system was developed to detect nodules (recall of 0.9507; FROC of 0.6470) and stratify the risk (ACC of 0.8696; macro-AUC of 0.8516) automatically. A novel model for lung cancer screening, the integration mobile CT with deep learning, was proposed. It enabled specialists to increase the accuracy and consistency of workflow and has potential to assist clinicians in detecting early-stage lung cancer effectively.

Fully automated segmentation of pneumonia infection based on Probabilistic Graphical Model and U-Net blend network

ABSTRACT COVID-19 spread rapidly in the global world, causing a serious medical treatment crisis. Automated segmentation of pulmonary infection from Computed Tomography (CT) images strengthened the traditional treatment strategy against COVID-19. We proposed an automated fully lung CT image infection segmentation framework named Probabilistic Graphical Model U-Net (PGM-U-Net). The whole framework iterative training end-to-end with general back-propagation mechanism with minor computational overhead from PGM component. The U-Net feature extractor benefits from modelling spatial correlations through the PGM component. Compared to the baseline network without modelling spatial correlations, experimental results illustrate that the proposed PGM-U-Net framework achieves higher accuracy probability maps of region predictions in the isolated infection regions. For further quantitative comparison experiment, we demonstrate that our framework outperforms the existing methods in pulmonary infection segmentation and achieves the Free-response Receiver Operating Characteristic Curve (FROC) score of 0.912 on the test data set.

A situational awareness Bayesian network approach for accurate and credible personalized adaptive radiotherapy outcomes prediction in lung cancer patients.

A situational awareness Bayesian network (SA-BN) approach is developed to improve physicians' trust in the prediction of radiation outcomes and evaluate its performance for personalized adaptive radiotherapy (pART). 118 non-small-cell lung cancer patients with their biophysical features were employed for discovery (n=68) and validation (n=50) of radiation outcomes prediction modeling. Patients' important characteristics identified by radiation experts to predict individual's tumor local control (LC) or radiation pneumonitis with grade≥2 (RP2) were incorporated as expert knowledge (EK). Besides generating an EK-based naïve BN (EK-NBN), an SA-BN was developed by incorporating the EK features into pure data-driven BN (PD-BN) methods to improve the credibility of LC or / and RP2 prediction. After using area under the free-response receiver operating characteristics curve (AU-FROC) to assess the joint prediction of these outcomes, their prediction performances were compared with a regression approach based on the expert yielded estimates (EYE) penalty and its variants. In addition to improving the credibility of radiation outcomes prediction, the SA-BN approach outperformed the EYE penalty and its variants in terms of the joint prediction of LC and RP2. The value of AU-FROC improves from 0.70 (95% CI: 0.54-0.76) using EK-NBN, to 0.75 (0.65-0.82) using a variant of EYE penalty, to 0.83 (0.75-0.93) using PD-BN and 0.83 (0.77-0.90) using SA-BN; with similar trends in the validation cohort. The SA-BN approach can provide an accurate and credible human-machine interface to gain physicians' trust in clinical decision-making, which has the potential to be an important component of pART.

Evaluation of data augmentation via synthetic images for improved breast mass detection on mammograms using deep learning.

We evaluated whether using synthetic mammograms for training data augmentation may reduce the effects of overfitting and increase the performance of a deep learning algorithm for breast mass detection. Synthetic mammograms were generated using in silico procedural analytic breast and breast mass modeling algorithms followed by simulated x-ray projections of the breast models into mammographic images. In silico breast phantoms containing masses were modeled across the four BI-RADS breast density categories, and the masses were modeled with different sizes, shapes, and margins. A Monte Carlo-based x-ray transport simulation code, MC-GPU, was used to project the three-dimensional phantoms into realistic synthetic mammograms. 2000 mammograms with 2522 masses were generated to augment a real data set during training. From the Curated Breast Imaging Subset of the Digital Database for Screening Mammography (CBIS-DDSM) data set, we used 1111 mammograms (1198 masses) for training, 120 mammograms (120 masses) for validation, and 361 mammograms (378 masses) for testing. We used faster R-CNN for our deep learning network with pretraining from ImageNet using the Resnet-101 architecture. We compared the detection performance when the network was trained using different percentages of the real CBIS-DDSM training set (100%, 50%, and 25%), and when these subsets of the training set were augmented with 250, 500, 1000, and 2000 synthetic mammograms. Free-response receiver operating characteristic (FROC) analysis was performed to compare performance with and without the synthetic mammograms. We generally observed an improved test FROC curve when training with the synthetic images compared to training without them, and the amount of improvement depended on the number of real and synthetic images used in training. Our study shows that enlarging the training data with synthetic samples can increase the performance of deep learning systems.

Performance of Deep Learning Model in Detecting Operable Lung Cancer With Chest Radiographs.

The aim of this study was to evaluate the diagnostic performance of a trained deep convolutional neural network (DCNN) model for detecting operable lung cancer with chest radiographs (CXRs). The institutional review board approved this study. A deep learning model (DLM) based on DCNN was trained with 17,211 CXRs (5700 CT-confirmed lung nodules in 3500 CXRs and 13,711 normal CXRs), finally augmented to 600,000 images. For validation, a trained DLM was tested with 1483 CXRs with surgically resected lung cancer, marked and scored by 2 radiologists. Furthermore, diagnostic performances of DLM and 6 human observers were compared with 500 cases (200 visible T1 lung cancer on CXR and 300 normal CXRs) and analyzed using free-response receiver-operating characteristics curve (FROC) analysis. The overall detection rate of DLM for resected lung cancers (27.2±14.6 mm) was a sensitivity of 76.8% (1139/1483) with a false positive per image (FPPI) of 0.3 and area under the FROC curve (AUC) of 0.732. In the comparison with human readers, DLM demonstrated a sensitivity of 86.5% at 0.1 FPPI and a sensitivity of 92% at 0.3 FPPI with AUC of 0.899 at an FPPI range of 0.03 to 0.44 for detecting visible T1 lung cancers, which were superior to the average of 6 human readers [mean sensitivity; 78% (range, 71.6% to 82.6%) at an FPPI of 0.1% and 85% (range, 80.2% to 89.2%) at an FPPI of 0.3, AUC of 0.819 (range, 0.754 to 0.862) at an FPPI of 0.03 to 0.44). A DLM has high diagnostic performance in detecting operable lung cancer with CXR, demonstrating a potential of playing a pivotal role for lung cancer screening.

Voxel-based morphometry analysis of double inversion-recovery magnetic resonance imaging for detecting microscopic lesions: a simulation study.

Double inversion-recovery (DIR) imaging has the potential to improve the detection of subcortical lesions through the use of voxel-based morphometry (VBM) analysis. The aim of this study was to clarify the characteristics of detectable lesions by performing a VBM analysis on DIR images of simulated lesions. Twenty healthy volunteers underwent magnetic resonance imaging using a head three-dimensional DIR sequence. The images were processed using SPM12; then, the selected images with simulated lesions were analyzed via VBM. The VBM results were evaluated using free-response receiver-operating characteristic curves and a receiver-operating characteristic analysis. The sensitivity was 100% (5/5), with 5.6 false-positive objects per case, in simulated lesions with a contrast of 0.6 and a size of 2.4mm. The sensitivity was 80% (4/5), with 5.4 false-positive objects per case, in simulated lesions with a contrast of 0.5 and a size of 2.4mm. The mean area under the curve value was increased from 0.783 to 0.883 using VBM, with a statistically significant difference (p < 0.01). The VBM analysis of the DIR images using SPM alone showed the potential to detect subcortical microscopic lesions. Early detection of Alzheimer's disease may be possible by adapting VBM in the clinical setting.

Development and Validation of a Deep Learning-Based Automated Detection Algorithm for Major Thoracic Diseases on Chest Radiographs

Background: Interpretation of chest radiographs (CRs) is a challenging task prone to errors, requiring expert readers. An automated system that can accurately classify CRs may help streamline the clinical workflow. Methods: We developed a deep-learning-based automatic detection (DLAD) algorithm for classification of CRs with major thoracic diseases including pulmonary malignancy, active pulmonary tuberculosis, pneumonia, and pneumothorax, using 54,221 normal CRs and 47,917 abnormal CRs, which were labeled and annotated by board certified radiologists. Performance of DLAD was validated using five independent datasets from different institutions. To compare the performances of DLAD with physicians, an observer performance test was conducted by 15 physicians including non-radiology physicians, board-certified radiologists, and thoracic radiologists. Image-wise classification performances and lesion-wise localization performances were measured by area under the receiver operating characteristic (ROC) curves, and area under the alternative free-response ROC curves, respectively. Findings: DLAD demonstrated classification performance of 0*973-1*000 and localization performance of 0*923-1*000 in independent datasets. DLAD demonstrated significantly higher performance than all three physician groups in both image-wise classification (0*983 vs. 0*814-0*932) and lesion-wise localization (0*985 vs. 0*781-0*907). Significant improvements in both image-wise classification (performance increment, 0*026-0*090) and lesion-wise localization (performance increment, 0*031-0*092) were observed in all three physician groups with assistance of the algorithm. Interpretation: Our DLAD demonstrated excellent and consistent performance in the detection of major thoracic diseases on CR, outperforming even thoracic radiologists. Funding Statement: The present study was supported by the Seoul National University Hospital Research fund (grant number: 04- 2016-3000), Lunit Inc. and the Seoul Research & Buisiness Development Program (grant number: FI170002). Declaration of Interests: Chang Min Park reports grants from Seoul National University Hospital, Seoul Metropolitan Government, and Lunit Inc., during the conduct of the study; Sunggyun Park and Jaehong Aum are employees of Lunit Inc. Ethics Approval Statement: The present study was approved by the institutional review boards of all participating institutions. Patients’ informed consents were waived.

Development and Validation of a Deep Learning-based Automatic Detection Algorithm for Active Pulmonary Tuberculosis on Chest Radiographs.

BackgroundDetection of active pulmonary tuberculosis on chest radiographs (CRs) is critical for the diagnosis and screening of tuberculosis. An automated system may help streamline the tuberculosis screening process and improve diagnostic performance.MethodsWe developed a deep learning–based automatic detection (DLAD) algorithm using 54c221 normal CRs and 6768 CRs with active pulmonary tuberculosis that were labeled and annotated by 13 board-certified radiologists. The performance of DLAD was validated using 6 external multicenter, multinational datasets. To compare the performances of DLAD with physicians, an observer performance test was conducted by 15 physicians including nonradiology physicians, board-certified radiologists, and thoracic radiologists. Image-wise classification and lesion-wise localization performances were measured using area under the receiver operating characteristic (ROC) curves and area under the alternative free-response ROC curves, respectively. Sensitivities and specificities of DLAD were calculated using 2 cutoffs (high sensitivity [98%] and high specificity [98%]) obtained through in-house validation.ResultsDLAD demonstrated classification performance of 0.977–1.000 and localization performance of 0.973–1.000. Sensitivities and specificities for classification were 94.3%–100% and 91.1%–100% using the high-sensitivity cutoff and 84.1%–99.0% and 99.1%–100% using the high-specificity cutoff. DLAD showed significantly higher performance in both classification (0.993 vs 0.746–0.971) and localization (0.993 vs 0.664–0.925) compared to all groups of physicians.ConclusionsOur DLAD demonstrated excellent and consistent performance in the detection of active pulmonary tuberculosis on CR, outperforming physicians, including thoracic radiologists.

Evaluation of diagnostic accuracy in free-response detection-localization tasks using ROC tools.

Diagnostic systems designed to detect possibly multiple lesions per patient (e.g. multiple polyps during CT colonoscopy) are often evaluated in "free-response" studies that allow for diagnostic responses unconstrained in their number and locations. Analysis of free-response studies requires extensions of the traditional receiver operating characteristic (ROC) analysis, which are termed free-response ROC (FROC) methodology. Despite substantial developments in this area, FROC tools and approaches are much more cumbersome than traditional ROC methods. Alternative approaches that use well-known ROC tools (e.g. ROI-ROC) require defining and physically delineating regions of interest (ROI) and combine FROC data within ROIs. We propose an approach that allows analyzing FROC data using conventional ROC tools without delineating the actual ROIs or reducing data. The design parameters of FROC study are used to make FROC data analyzable using ROC tools and to calibrate the corresponding FROC and ROC curves on both conceptual and numerical levels. Differences in the performance indices of the nonparametric FROC and the new approach are shown to be asymptotically negligible and typically rather small in practice. Data from a large multi-reader study of colon cancer detection are used to illustrate the new approach.

A context-sensitive deep learning approach for microcalcification detection in mammograms

A challenging issue in computerized detection of clustered microcalcifications (MCs) is the frequent occurrence of false positives (FPs) caused by local image patterns that resemble MCs. We develop a context-sensitive deep neural network (DNN), aimed to take into account both the local image features of an MC and its surrounding tissue background, for MC detection. The DNN classifier is trained to automatically extract the relevant image features of an MC as well as its image context. The proposed approach was evaluated on a set of 292 mammograms using free-response receiver operating characteristic (FROC) analysis on the accuracy both in detecting individual MCs and in detecting MC clusters. The results demonstrate that the proposed approach could achieve significantly higher FROC curves when compared to two MC-based detectors. It indicates that incorporating image context information in MC detection can be beneficial for reducing the FPs in detections.

Analysis of computer-aided detection techniques and signal characteristics for clustered microcalcifications on digital mammography and digital breast tomosynthesis

With IRB approval, digital breast tomosynthesis (DBT) images of human subjects were collected using a GE GEN2 DBT prototype system. Corresponding digital mammograms (DMs) of the same subjects were collected retrospectively from patient files. The data set contained a total of 237 views of DBT and equal number of DM views from 120 human subjects, each included 163 views with microcalcification clusters (MCs) and 74 views without MCs. The data set was separated into training and independent test sets. The pre-processing, object prescreening and segmentation, false positive reduction and clustering strategies for MC detection by three computer-aided detection (CADe) systems designed for DM, DBT, and a planar projection image generated from DBT were analyzed. Receiver operating characteristic (ROC) curves based on features extracted from microcalcifications and free-response ROC (FROC) curves based on scores from MCs were used to quantify the performance of the systems. Jackknife FROC (JAFROC) and non-parametric analysis methods were used to determine the statistical difference between the FROC curves. The difference between the CADDM and CADDBT systems when the false positive rate was estimated from cases without MCs did not reach statistical significance. The study indicates that the large search space in DBT may not be a limiting factor for CADe to achieve similar performance as that observed in DM.

Technical Note: Image filtering to make computer-aided detection robust to image reconstruction kernel choice in lung cancer CT screening.

In lung cancer computed tomography (CT) screening, the performance of a computer-aided detection (CAD) system depends on the selection of the image reconstruction kernel. To reduce this dependence on reconstruction kernels, the authors propose a novel application of an image filtering method previously proposed by their group. The proposed filtering process uses the ratio of modulation transfer functions (MTFs) of two reconstruction kernels as a filtering function in the spatial-frequency domain. This method is referred to as MTFratio filtering. Test image data were obtained from CT screening scans of 67 subjects who each had one nodule. Images were reconstructed using two kernels: fSTD (for standard lung imaging) and fSHARP (for sharp edge-enhancement lung imaging). The MTFratio filtering was implemented using the MTFs measured for those kernels and was applied to the reconstructed fSHARP images to obtain images that were similar to the fSTD images. A mean filter and a median filter were applied (separately) for comparison. All reconstructed and filtered images were processed using their prototype CAD system. The MTFratio filtered images showed excellent agreement with the fSTD images. The standard deviation for the difference between these images was very small, ∼6.0 Hounsfield units (HU). However, the mean and median filtered images showed larger differences of ∼48.1 and ∼57.9 HU from the fSTD images, respectively. The free-response receiver operating characteristic (FROC) curve for the fSHARP images indicated poorer performance compared with the FROC curve for the fSTD images. The FROC curve for the MTFratio filtered images was equivalent to the curve for the fSTD images. However, this similarity was not achieved by using the mean filter or median filter. The accuracy of MTFratio image filtering was verified and the method was demonstrated to be effective for reducing the kernel dependence of CAD performance.

Efficacy of automated computer-aided diagnosis of retinal nerve fibre layer defects in healthcare screening.

To evaluate the efficacy of a new automatic computer-aided detection (CAD) system for mass screening of retinal nerve fibre layer (RNFL) defects in a large population using fundus photographs. Among the fundus photographs of 1200 consecutive subjects who visited a healthcare centre, a total of 2270 photographs appropriate for analysis were tested. The photographs were first reviewed by two expert ophthalmologists for detection of RNFL defects (gold standard manual detection). The images were then analysed using an automatic CAD system for detecting RNFL defects in various cases of glaucomatous and non-glaucomatous optic neuropathy. A free-response receiver operating characteristics curve was generated to evaluate the validity of the CAD system. The results of the automatic detection were compared with those of manual detection, and sensitivity and specificity of the CAD system were calculated. In manual detection of 2270 photographs, 41 RNFL defects from 36 photographs (1.6%) were detected, and no RNFL defects were found in 2234 photographs (98.4%). The sensitivity of the CAD system for detecting RNFL defects was 90.2% (37/41 RNFL defects) and the specificity was 72.5% (1620/2234 photographs with no RNFL defects) at a false-positive rate of 0.36 per image. The new CAD system successfully detected RNFL defects during mass screening of fundus photographs in a large population who visited a healthcare centre. The proposed algorithm can be useful for clinicians in screening RNFL defects in healthcare centres. The false-positive rate is still unsatisfactory, although improved compared with the previous study. Further studies are needed to enhance the speed and specificity of image analysis using the CAD system.

Optimal Joint Detection and Estimation That Maximizes ROC-Type Curves.

Combined detection-estimation tasks are frequently encountered in medical imaging. Optimal methods for joint detection and estimation are of interest because they provide upper bounds on observer performance, and can potentially be utilized for imaging system optimization, evaluation of observer efficiency, and development of image formation algorithms. We present a unified Bayesian framework for decision rules that maximize receiver operating characteristic (ROC)-type summary curves, including ROC, localization ROC (LROC), estimation ROC (EROC), free-response ROC (FROC), alternative free-response ROC (AFROC), and exponentially-transformed FROC (EFROC) curves, succinctly summarizing previous results. The approach relies on an interpretation of ROC-type summary curves as plots of an expected utility versus an expected disutility (or penalty) for signal-present decisions. We propose a general utility structure that is flexible enough to encompass many ROC variants and yet sufficiently constrained to allow derivation of a linear expected utility equation that is similar to that for simple binary detection. We illustrate our theory with an example comparing decision strategies for joint detection-estimation of a known signal with unknown amplitude. In addition, building on insights from our utility framework, we propose new ROC-type summary curves and associated optimal decision rules for joint detection-estimation tasks with an unknown, potentially-multiple, number of signals in each observation.