Free-response Receiver Operating Characteristic Research Articles

3D printed phantom with 12 000 submillimeter lesions to improve efficiency in CT detectability assessment.

The detectability performance of a CT scanner is difficult to precisely quantify when nonlinearities are present in reconstruction. An efficient detectability assessment method that is sensitive to small effects of dose and scanner settings is desirable. We previously proposed a method using a search challenge instrument: a phantom is embedded with hundreds of lesions at random locations, and a model observer is used to detect lesions. Preliminary tests in simulation and a prototype showed promising results. In this work, we fabricated a full-size search challenge phantom with design updates, including changes to lesion size, contrast, and number, and studied our implementation by comparing the lesion detectability from a nonprewhitening (NPW) model observer between different reconstructions at different exposure levels, and by estimating the instrument sensitivity to detect changes in dose. Designed to fit into QRM anthropomorphic phantoms, our search challenge phantom is a cylindrical insert 10cm wide and 4cm thick, embedded with 12 000 lesions (nominal width of 0.6mm, height of 0.8mm, and contrast of -350 HU), and was fabricated using PixelPrint, a 3D printing technique. The insert was scanned alone at a high dose to assess printing accuracy. To evaluate lesion detectability, the insert was placed in a QRM thorax phantom and scanned from 50 to 625 mAs with increments of 25 mAs, once per exposure level, and the average of all exposure levels was used as high-dose reference. Scans were reconstructed with three different settings: filtered-backprojection (FBP) with Br40 and Br59, and Sinogram Affirmed Iterative Reconstruction (SAFIRE) with strength level 5 and Br59 kernel. An NPW model observer was used to search for lesions, and detection performance of different settings were compared using area under the exponential transform of free response ROC curve (AUC). Using propagation of uncertainty, the sensitivity to changes in dose was estimated by the percent change in exposure due to one standard deviation of AUC, measured from 5 repeat scans at 100, 200, 300, and 400 mAs. The printed insert lesions had an average position error of 0.20mm compared to printing reference. As the exposure level increases from 50 mAs to 625 mAs, the lesion detectability AUCs increase from 0.38 to 0.92, 0.42 to 0.98, and 0.41 to 0.97 for FBP Br40, FBP Br59, and SAFIRE Br59, respectively, with a lower rate of increase at higher exposure level. FBP Br59 performed best with AUC 0.01 higher than SAFIRE Br59 on average and 0.07 higher than FBP Br40 (all P<0.001). The standard deviation of AUC was less than 0.006, and the sensitivity to detect changes in mAs was within 2% for FBP Br59. Our 3D-printed search challenge phantom with 12 000 submillimeter lesions, together with an NPW model observer, provide an efficient CT detectability assessment method that is sensitive to subtle effects in reconstruction and is sensitive to small changes in dose.

Part II: Effect of different evaluation methods to the application of a computer-aided prostate MRI detection/diagnosis (CADe/CADx) device on reader performance

IntroductionThe construction and results of a multiple-reader multiple-case prostate MRI study are described and reported to illustrate recommendations for how to standardize artificial intelligence (AI) prostate studies per the review constituting Part I1. MethodsOur previously reported approach was applied to review and report an IRB approved, HIPAA compliant multiple-reader multiple-case clinical study of 150 bi-parametric prostate MRI studies across 9 readers, measuring physician performance both with and without the use of the recently FDA cleared CADe/CADx software ProstatID. ResultsUnassisted reader AUC values ranged from 0.418 – 0.759, with AI assisted AUC values ranging from 0.507 – 0.787. This represented a statistically significant AUC improvement of 0.045 (α = 0.05). A free-response ROC (FROC) analysis similarly demonstrated a statistically significant increase in θ from 0.405 to 0.453 (α = 0.05). The standalone performance of ProstatID performed across all prostate tissues demonstrated an AUC of 0.929, while the standalone lesion level performance of ProstatID at all biopsied locations achieved an AUC of 0.710. ConclusionThis study applies and illustrates suggested reporting and standardization methods for prostate AI studies that will make it easier to understand, evaluate and compare between AI studies. Providing radiologists with the ProstatID CADe/CADx software significantly increased diagnostic performance as assessed by both ROC and free-response ROC metrics. Such algorithms have the potential to improve radiologist performance in the detection and localization of clinically significant prostate cancer.

Combining public datasets for automated tooth assessment in panoramic radiographs

ObjectivePanoramic radiographs (PRs) provide a comprehensive view of the oral and maxillofacial region and are used routinely to assess dental and osseous pathologies. Artificial intelligence (AI) can be used to improve the diagnostic accuracy of PRs compared to bitewings and periapical radiographs. This study aimed to evaluate the advantages and challenges of using publicly available datasets in dental AI research, focusing on solving the novel task of predicting tooth segmentations, FDI numbers, and tooth diagnoses, simultaneously.Materials and methodsDatasets from the OdontoAI platform (tooth instance segmentations) and the DENTEX challenge (tooth bounding boxes with associated diagnoses) were combined to develop a two-stage AI model. The first stage implemented tooth instance segmentation with FDI numbering and extracted regions of interest around each tooth segmentation, whereafter the second stage implemented multi-label classification to detect dental caries, impacted teeth, and periapical lesions in PRs. The performance of the automated tooth segmentation algorithm was evaluated using a free-response receiver-operating-characteristics (FROC) curve and mean average precision (mAP) metrics. The diagnostic accuracy of detection and classification of dental pathology was evaluated with ROC curves and F1 and AUC metrics.ResultsThe two-stage AI model achieved high accuracy in tooth segmentations with a FROC score of 0.988 and a mAP of 0.848. High accuracy was also achieved in the diagnostic classification of impacted teeth (F1 = 0.901, AUC = 0.996), whereas moderate accuracy was achieved in the diagnostic classification of deep caries (F1 = 0.683, AUC = 0.960), early caries (F1 = 0.662, AUC = 0.881), and periapical lesions (F1 = 0.603, AUC = 0.974). The model’s performance correlated positively with the quality of annotations in the used public datasets. Selected samples from the DENTEX dataset revealed cases of missing (false-negative) and incorrect (false-positive) diagnoses, which negatively influenced the performance of the AI model.ConclusionsThe use and pooling of public datasets in dental AI research can significantly accelerate the development of new AI models and enable fast exploration of novel tasks. However, standardized quality assurance is essential before using the datasets to ensure reliable outcomes and limit potential biases.

Development and multicenter validation of deep convolutional neural network-based detection of colorectal cancer on abdominal CT.

This study aims to develop computer-aided detection (CAD) for colorectal cancer (CRC) using abdominal CT based on a deep convolutional neural network. This retrospective study included consecutive patients with colorectal adenocarcinoma who underwent abdominal CT before CRC resection surgery (training set = 379, test set = 103). We customized the 3D U-Net of nnU-Net (CUNET) for CRC detection, which was trained with fivefold cross-validation using annotated CT images. CUNET was validated using datasets covering various clinical situations and institutions: an internal test set (n = 103), internal patients with CRC first determined by CT (n = 54) and asymptomatic CRC (n = 51), and an external validation set from two institutions (n = 60). During each validation, data from the healthy population were added (internal = 60; external = 130). CUNET was compared with other deep CNNs: residual U-Net and EfficientDet. The CAD performances were evaluated using per-CRC sensitivity (true positive/all CRCs), free-response receiver operating characteristic (FROC), and jackknife alternative FROC (JAFROC) curves. CUNET showed a higher maximum per-CRC sensitivity than residual U-Net and EfficientDet (internal test set 91.3% vs. 61.2%, and 64.1%). The per-CRC sensitivity of CUNET at false-positive rates of 3.0 was as follows: internal CRC determined by CT, 89.3%; internal asymptomatic CRC, 87.3%; and external validation, 89.6%. CUNET detected 69.2% (9/13) of CRCs missed by radiologists and 89.7% (252/281) of CRCs from all validation sets. CUNET can detect CRC on abdominal CT in patients with various clinical situations and from external institutions. • Customized 3D U-Net of nnU-Net (CUNET) can be applied to the opportunistic detection of colorectal cancer (CRC) in abdominal CT, helping radiologists detect unexpected CRC. • CUNET showed the best performance at false-positive rates ≥ 3.0, and 30.1% of false-positives were in the colorectum. CUNET detected 69.2% (9/13) of CRCs missed by radiologists and 87.3% (48/55) of asymptomatic CRCs. • CUNET detected CRCs in multiple validation sets composed of varying clinical situations and from different institutions, and CUNET detected 89.7% (252/281) of CRCs from all validation sets.

Assessing clinical efficacy of polyp detection models using open-access datasets.

Ensuring accurate polyp detection during colonoscopy is essential for preventing colorectal cancer (CRC). Recent advances in deep learning-based computer-aided detection (CADe) systems have shown promise in enhancing endoscopists' performances. Effective CADe systems must achieve high polyp detection rates from the initial seconds of polyp appearance while maintaining low false positive (FP) detection rates throughout the procedure. We integrated four open-access datasets into a unified platform containing over 340,000 images from various centers, including 380 annotated polyps, with distinct data splits for comprehensive model development and benchmarking. The REAL-Colon dataset, comprising 60 full-procedure colonoscopy videos from six centers, is used as the fifth dataset of the platform to simulate clinical conditions for model evaluation on unseen center data. Performance assessment includes traditional object detection metrics and new metrics that better meet clinical needs. Specifically, by defining detection events as sequences of consecutive detections, we compute per-polyp recall at early detection stages and average per-patient FPs, enabling the generation of Free-Response Receiver Operating Characteristic (FROC) curves. Using YOLOv7, we trained and tested several models across the proposed data splits, showcasing the robustness of our open-access platform for CADe system development and benchmarking. The introduction of new metrics allows for the optimization of CADe operational parameters based on clinically relevant criteria, such as per-patient FPs and early polyp detection. Our findings also reveal that omitting full-procedure videos leads to non-realistic assessments and that detecting small polyp bounding boxes poses the greatest challenge. This study demonstrates how newly available open-access data supports ongoing research progress in environments that closely mimic clinical settings. The introduced metrics and FROC curves illustrate CADe clinical efficacy and can aid in tuning CADe hyperparameters.

Ipsilateral Lesion Detection Refinement for Tomosynthesis.

Computer-aided detection (CAD) frameworks for breast cancer screening have been researched for several decades. Early adoption of deep-learning models in CAD frameworks has shown greatly improved detection performance compared to traditional CAD on single-view images. Recently, studies have improved performance by merging information from multiple views within each screening exam. Clinically, the integration of lesion correspondence during screening is a complicated decision process that depends on the correct execution of several referencing steps. However, most multi-view CAD frameworks are deep-learning-based black-box techniques. Fully end-to-end designs make it very difficult to analyze model behaviors and fine-tune performance. More importantly, the black-box nature of the techniques discourages clinical adoption due to the lack of explicit reasoning for each multi-view referencing step. Therefore, there is a need for a multi-view detection framework that can not only detect cancers accurately but also provide step-by-step, multi-view reasoning. In this work, we present Ipsilateral-Matching-Refinement Networks (IMR-Net) for digital breast tomosynthesis (DBT) lesion detection across multiple views. Our proposed framework adaptively refines the single-view detection scores based on explicit ipsilateral lesion matching. IMR-Net is built on a robust, single-view detection CAD pipeline with a commercial development DBT dataset of 24675 DBT volumetric views from 8034 exams. Performance is measured using location-based, case-level receiver operating characteristic (ROC) and case-level free-response ROC (FROC) analysis.

A semi-supervised learning-based quality evaluation system for digital chest radiographs.

Digital radiography is the most commonly utilized medical imaging technique worldwide, and the quality of radiographs plays a crucial role in accurate disease diagnosis. Therefore, evaluating the quality of radiographs is an essential step in medical examinations. However, manual evaluation can be time-consuming, labor-intensive, and prone to interobserver differences, making it less reliable. To alleviate the workload of radiographic technologists and enhance the efficiency of radiograph quality evaluation, it is crucial to develop rapid and reliable quality evaluation methods and establish a set of quantitative evaluation standards. To address this, we have proposed a quality evaluation system for digital radiographs that utilizes deep learning techniques to achieve fast and precise evaluation. The evaluation of frontal chest radiograph quality involves assessing patient positioning through semantic segmentation and foreign body detection. For lung, scapula, and clavicle segmentation in digital chest radiographs, a residual connection-based convolutional neural network π-ResUNet, was proposed. Criteria for patient positioning evaluation were established based on the segmentation and manual evaluation results. A convolutional neural network, FasterRCNN, was utilized to detect and localize foreign bodies in digital chest radiographs. To enhance the performance of both neural networks, a semi-supervised learning (SSL) strategy was implemented by incorporating a consistency loss that leverages a large number of unlabeled digital radiographs. We also trained the network using the fully supervised learning (FSL) strategy and compared their performance on the test set. The ChestXRay-14 and object-CXR datasets were used throughout the process. By comparing with the manual annotation, the proposed network, trained using the SSL method, achieved a high Dice similarity coefficient (DSC) of 0.96, 0.88, and 0.88 for lung, scapula, and clavicle segmentation, respectively, outperforming the network trained with the FSL method. In addition, for foreign body detection, the proposed SSL method was superior to the FSL method, achieving an AUC (Area under receiver operating characteristic curve, Area under ROC curve) of 0.90 and an FROC (Free-response ROC) of 0.77 on the test dataset. The experimental results show that our proposed system is well-suited for radiograph quality evaluation, with the semi-supervised learning method further improving the network's performance. The proposed method can evaluate the quality of a chest radiograph from two aspects-patient positioning and foreign body detection-within 1 s, offering a promising tool in radiograph quality evaluation.

Semisupervised Learning with Report-guided Pseudo Labels for Deep Learning-based Prostate Cancer Detection Using Biparametric MRI.

To evaluate a novel method of semisupervised learning (SSL) guided by automated sparse information from diagnostic reports to leverage additional data for deep learning-based malignancy detection in patients with clinically significant prostate cancer. This retrospective study included 7756 prostate MRI examinations (6380 patients) performed between January 2014 and December 2020 for model development. An SSL method, report-guided SSL (RG-SSL), was developed for detection of clinically significant prostate cancer using biparametric MRI. RG-SSL, supervised learning (SL), and state-of-the-art SSL methods were trained using 100, 300, 1000, or 3050 manually annotated examinations. Performance on detection of clinically significant prostate cancer by RG-SSL, SL, and SSL was compared on 300 unseen examinations from an external center with a histopathologically confirmed reference standard. Performance was evaluated using receiver operating characteristic (ROC) and free-response ROC analysis. P values for performance differences were generated with a permutation test. At 100 manually annotated examinations, mean examination-based diagnostic area under the ROC curve (AUC) values for RG-SSL, SL, and the best SSL were 0.86 ± 0.01 (SD), 0.78 ± 0.03, and 0.81 ± 0.02, respectively. Lesion-based detection partial AUCs were 0.62 ± 0.02, 0.44 ± 0.04, and 0.48 ± 0.09, respectively. Examination-based performance of SL with 3050 examinations was matched by RG-SSL with 169 manually annotated examinations, thus requiring 14 times fewer annotations. Lesion-based performance was matched with 431 manually annotated examinations, requiring six times fewer annotations. RG-SSL outperformed SSL in clinically significant prostate cancer detection and achieved performance similar to SL even at very low annotation budgets.Keywords: Annotation Efficiency, Computer-aided Detection and Diagnosis, MRI, Prostate Cancer, Semisupervised Deep Learning Supplemental material is available for this article. Published under a CC BY 4.0 license.

Breast cancer detection with upstream data fusion, machine learning, and automated registration: initial results.

To develop an artificial intelligence algorithm for the detection of breast cancer by combining upstream data fusion (UDF), machine learning (ML), and automated registration, using digital breast tomosynthesis (DBT) and breast ultrasound (US). Our retrospective study included examinations from 875 women obtained between April 2013 and January 2019. Included patients had a DBT mammogram, breast US, and biopsy proven breast lesion. Images were annotated by a breast imaging radiologist. An AI algorithm was developed based on ML for image candidate detections and UDF for fused detections. After exclusions, images from 150 patients were evaluated. Ninety-five cases were used for training and validation of ML. Fifty-five cases were included in the UDF test set. UDF performance was evaluated with a free-response receiver operating characteristic (FROC) curve. Forty percent of cases evaluated with UDF (22/55) yielded true ML detections in all three images (craniocaudal DBT, mediolateral oblique DBT, and US). Of these, 20/22 (90.9%) produced a UDF fused detection that contained and classified the lesion correctly. FROC analysis for these cases showed 90% sensitivity at 0.3 false positives per case. In contrast, ML yielded an average of 8.0 false alarms per case. An AI algorithm combining UDF, ML, and automated registration was developed and applied to test cases, showing that UDF can yield fused detections and decrease false alarms when applied to breast cancer detection. Improvement of ML detection is needed to realize the full benefit of UDF.

Automatic detection of prostate cancer grades and chronic prostatitis in biparametric MRI

Background and objective:With emerging evidence to improve prostate cancer (PCa) screening, multiparametric magnetic prostate imaging is becoming an essential noninvasive component of the diagnostic routine. Computer-aided diagnostic (CAD) tools powered by deep learning can help radiologists interpret multiple volumetric images. In this work, our objective was to examine promising methods recently proposed in the multigrade prostate cancer detection task and to suggest practical considerations regarding model training in this context.Methods:We collected 1647 fine-grained biopsy-confirmed findings, including Gleason scores and prostatitis, to form a training dataset. In our experimental framework for lesion detection, all models utilized 3D nnU-Net architecture that accounts for anisotropy in the MRI data. First, we explore an optimal range of b-values for diffusion-weighted imaging (DWI) modality and its effect on the detection of clinically significant prostate cancer (csPCa) and prostatitis using deep learning, as the optimal range is not yet clearly defined in this domain. Next, we propose a simulated multimodal shift as a data augmentation technique to compensate for the multimodal shift present in the data. Third, we study the effect of incorporating the prostatitis class alongside cancer-related findings at three different granularities of the prostate cancer class (coarse, medium, and fine) and its impact on the detection rate of the target csPCa. Furthermore, ordinal and one-hot encoded (OHE) output formulations were tested.Results: An optimal model configuration with fine class granularity (prostatitis included) and OHE has scored the lesion-wise partial Free-Response Receiver Operating Characteristic (FROC) area under the curve (AUC) of 1.94 (CI 95%: 1.76–2.11) and patient-wise ROC AUC of 0.874 (CI 95%: 0.793–0.938) in the detection of csPCa. Inclusion of the auxiliary prostatitis class has demonstrated a stable relative improvement in specificity at a false positive rate (FPR) of 1.0 per patient, with an increase of 3%, 7%, and 4% for coarse, medium, and fine class granularities.Conclusions: This paper examines several configurations for model training in the biparametric MRI setup and proposes optimal value ranges. It also shows that the fine-grained class configuration, including prostatitis, is beneficial for detecting csPCa. The ability to detect prostatitis in all low-risk cancer lesions suggests the potential to improve the quality of the early diagnosis of prostate diseases. It also implies an improved interpretability of the results by the radiologist.

Improvements in dose efficiency with high resolution scan modes in photon counting CT.

For the detection of very small objects, high resolution detectors are expected to provide higher dose efficiency. We assessed this impact of increased resolution on a clinical photon counting detector CT (PCD-CT) by comparing its detectability in high resolution and standard resolution (with 2×2 binning and larger focal spot) modes. A 50μm-thin metal wire was placed in a thorax phantom and scanned in both modes at three exposure levels (12, 15, and 18 mAs); acquired data were reconstructed with three reconstruction kernels (Br40, Br68, and Br76, from smooth to sharp). A scanning nonprewhitening model observer searched for the wire location within each slice independently. Detection performance was quantified as area under the exponential transform of the free response ROC curve. The high-resolution mode had the mean AUCs at 18 mAs of 0.45, 0.49, and 0.65 for Br40, Br68, and Br76, respectively, which were 2 times, 3.6 times, and 4.6 times those of the standard resolution mode. The high-resolution mode achieved greater AUC at 12 mAs than the standard resolution mode at 18 mAs for every reconstruction kernel, but improvements were larger at sharper kernels. The results are consistent with the greater suppression of noise aliasing expected at higher frequencies with high resolution CT. This work illustrates that PCD-CT can provide large dose efficiency gains for detection tasks of small, high contrast lesions.

Fully-automated detection of small bowel carcinoid tumors in CT scans using deep learning.

Small bowel carcinoid tumor is a rare neoplasm and increasing in incidence. Patients with small bowel carcinoid tumors often experience long delays in diagnosis due to the vague symptoms, slow growth of tumors, and lack of clinician awareness. Computed tomography (CT) is the most common imaging study for diagnosis of small bowel carcinoid tumor. It is often used with positron emission tomography (PET) to capture anatomical and functional aspects of carcinoid tumors and thus to increase thesensitivity. We compared three different kinds of methods for the automatic detection of small bowel carcinoid tumors on CT scans, which is the first to the best of ourknowledge. Thirty-three preoperative CT scans of 33 unique patients with surgically-proven carcinoid tumors within the small bowel were collected. Ground-truth segmentation of tumors was drawn on CT scans by referring to available 18 F-DOPA PET scans and the corresponding radiology report. These scans were split into the trainval set (n = 24) and the test positive set (n= 9). Additionally, 22 CT scans of 22 unique patients who had no evidence of the tumor were collected to comprise the test negative set. We compared three different kinds of detection methods, which are detection network, patch-based classification, and segmentation-based methods. We also investigated the usefulness of small bowel segmentation for reduction of false positives (FPs) for each method. Free-response receiver operating characteristic (FROC) curves and receiver operating characteristic (ROC) curves were used for lesion- and patient-level evaluations, respectively. Statistical analyses comparing the FROC and ROC curves were alsoperformed. The detection network method performed the best among the compared methods. For lesion-level detection, the detection network method, without the small bowel segmentation-based filtering, achieved sensitivity values of (60.8%, 81.1%, 82.4%, 86.5%) at per-scan FP rates of (1, 2, 4 ,8), respectively. The use of the small bowel segmentation did not improve the performance ( ). For patient-level detection, again the detection network method, but with the small bowel segmentation-based filtering, achieved the highest AUC of 0.86 with a sensitivity of 78% and specificity of 82% at the Youdenpoint. The carcinoid tumors in this patient population were very small and potentially difficult to diagnose. The presented method showed reasonable sensitivity at small numbers of FPs for lesion-level detection. It also achieved a promising AUC for patient-level detection. The method may have clinical application in patients with this rare and difficult to detectdisease.

Automatic detection of microaneurysms in fundus images based on multiple preprocessing fusion to extract features

Diabetic retinopathy (DR) is the main cause of new cases of blindness, and retinal microaneurysm (MA) is one of the early clinical manifestations of DR. However, it is difficult to directly observe MA with naked eyes because of their small size and poor contrast with the surrounding background. This paper presents a new method for automatic detection of MAs from fundus images. It mainly includes four steps: image preprocessing, candidate region extraction, feature extraction and classification. At the stage of feature extraction, different from traditional methods, we propose a feature extraction method based on multi-preprocessing fusion. In this paper, features are extracted from the original green channel image and the original gray image respectively, and then different processing methods are used to extract the same features from multiple pre-processed images. Finally, the extracted features are classified by multi-layer perceptron (MLP) classifier to determine whether the candidate object is microaneurysm (MA). The method is evaluated on e-ophtha-MA database and achieved high sensitivity. Using the FROC (free-response ROC) indicator, the detection achieves the F-score, accuracy, sensitivity, specificity and AUC (area under the curve) of 0.507, 0.940, 0.870, 0.938 and 0.9821, respectively, compared to the most advanced methods available. The proposed method of multi-preprocessing and fusion feature extraction is beneficial to classification and improves the detection performance.

Variations of image interpretations of radiologists from different populations in mammography and tomosynthesis with different levels of breast density.

This study aims to investigate the diagnostic performances of Australian and Shanghai-based Chinese radiologists in reading full-field digital mammogram (FFDM) and digital breast tomosynthesis (DBT) with different levels of breast density. Eighty-two Australian radiologists interpreted a 60-case FFDM set, and 29 radiologists also reported a 35-case DBT set. Sixty Shanghai radiologists read the same FFDM set, and 32 radiologists read the DBT set. The diagnostic performances of Australian and Shanghai radiologists were assessed using truth data (cancer cases were biopsy proven) and compared overall in specificity, case sensitivity, lesion sensitivity, receiver operating characteristics (ROC) area under the curve, and jack-knife free-response receiver operating characteristics (JAFROC) figure of merit, and they were stratified by case characteristics using the Mann-Whitney U test. The Spearman rank test was used to explore the association between radiologists' performances and their work experience in mammogram interpretation. There were significantly higher performances of Australian radiologists compared with Shanghai radiologists in low breast density for case sensitivity, lesion sensitivity, ROC, and JAFROC in the FFDM set ( ); in high breast density, Shanghai radiologists' performances in lesion sensitivity and JAFROC were also lower than Australian radiologists ( ). In the DBT test set, Australian radiologists performed better than Shanghai radiologists in cancer detection in both low and high breast density. The work experience of Australian radiologists was positively linked to their diagnostic performances, whereas this association was not statistically significant in Shanghai radiologists. There were significant variations in reading performances between Australian and Shanghai radiologists in FFDM and DBT across different levels of breast density, lesion types, and lesion sizes. An effective training initiative tailored to suit local readers is essential to enhancing the diagnostic accuracy of Shanghai radiologists.

Lung nodule detection in chest CT images based on vision transformer network with Bayesian optimization

PurposeLung cancer patients are usually diagnosed by the Computed tomography (CT) modality with the help of radiologists. Frequently, radiologists manually analyse CT images, which is a big task that consumes more time and results in inaccuracy. Therefore, this study proposes an automated computer-aided diagnosis (CAD) scheme based on Vision Transformer architecture with Bayesian Optimisation for lung nodule detection to assist radiologists in decision-making. MethodsThe Vision Transformer architecture relies on the shifted window-based vision transformer encoders to improve the processing and propagation efficiency of the lung nodule features. The baseline detection model is developed by integrating shifted window-based vision transformer for deep nodule feature extractions and a region proposal network to predict candidate nodules with low network complexity. The Bayesian Optimisation with the Gaussian method was used to optimise the hyperparameter values from the baseline model suitable for lung nodule detection on chest CT images. ResultsWe used 888 CT images with 10-fold cross-validation from the Lung Nodule Analysis 2016 (LUNA16) public dataset to validate the proposed network. Free-response receiver operating characteristics and competition performance metrics (CPM) was used for the performance assessment. The proposed architecture obtained the highest detection sensitivity of 98.39% and the CPM score of 0.909 with few network parameters. ConclusionThis study proposes a CAD system based on a vision transformer network determined by Bayesian Optimisation for lung nodule detection in chest CT images. The empirical results prove the proposed algorithm's effectiveness compared to the state-of-the-art methods.

Weakly supervised breast lesion detection in DCE-MRI using self-transfer learning.

Breast cancer is a typically diagnosed and life-threatening cancer in women. Thus, dynamic contrast-enhanced magnetic resonance imaging (DCE-MRI) is increasingly used for breast lesion detection and diagnosis because of the high resolution of soft tissues. Moreover, supervised detection methods have been implemented for breast lesion detection. However, these methods require substantial time and specialized staff to develop the labeled training samples. To investigate the potential of weakly supervised deep learning models for breast lesion detection. A total of 1003 breast DCE-MRI studies were collected, including 603 abnormal cases with 770 breast lesions and 400 normal subjects. The proposed model was trained using breast DCE-MRI considering only the image-level labels (normal and abnormal) and optimized for classification and detection sub-tasks simultaneously. Ablation experiments were performed to evaluate different convolutional neural network (CNN) backbones (VGG19 and ResNet50) as shared convolutional layers, as well as to evaluate the effect of the preprocessing methods. Our weakly supervised model performed better with VGG19 than with ResNet50 (p < 0.05). The average precision (AP) of the classification sub-task was 91.7% for abnormal cases and 88.0% for normal samples. The area under the receiver operating characteristic (ROC) curve (AUC) was 0.939 (95% confidence interval [CI]: 0.920-0.941). The weakly supervised detection task AP was 85.7%, and the correct location (CorLoc) was 90.2%. A sensitivity of 84.0% at two-false positives per image was assessed based on free-response ROC (FROC) curve. The results confirm that a weakly supervised CNN based on self-transfer learning is an effective and promising auxiliary tool for detecting breast lesions.

Convolutional neural network for detecting rib fractures on chest radiographs: a feasibility study

BackgroundChest radiography is the standard investigation for identifying rib fractures. The application of artificial intelligence (AI) for detecting rib fractures on chest radiographs is limited by image quality control and multilesion screening. To our knowledge, few studies have developed and verified the performance of an AI model for detecting rib fractures by using multi-center radiographs. And existing studies using chest radiographs for multiple rib fracture detection have used more complex and slower detection algorithms, so we aimed to create a multiple rib fracture detection model by using a convolutional neural network (CNN), based on multi-center and quality-normalised chest radiographs.MethodsA total of 1080 radiographs with rib fractures were obtained and randomly divided into the training set (918 radiographs, 85%) and the testing set (162 radiographs, 15%). An object detection CNN, You Only Look Once v3 (YOLOv3), was adopted to build the detection model. Receiver operating characteristic (ROC) and free-response ROC (FROC) were used to evaluate the model’s performance. A joint testing group of 162 radiographs with rib fractures and 233 radiographs without rib fractures was used as the internal testing set. Furthermore, an additional 201 radiographs, 121 with rib fractures and 80 without rib fractures, were independently validated to compare the CNN model performance with the diagnostic efficiency of radiologists.ResultsThe sensitivity of the model in the training and testing sets was 92.0% and 91.1%, respectively, and the precision was 68.0% and 81.6%, respectively. FROC in the testing set showed that the sensitivity for whole-lesion detection reached 91.3% when the false-positive of each case was 0.56. In the joint testing group, the case-level accuracy, sensitivity, specificity, and area under the curve were 85.1%, 93.2%, 79.4%, and 0.92, respectively. At the fracture level and the case level in the independent validation set, the accuracy and sensitivity of the CNN model were always higher or close to radiologists’ readings.ConclusionsThe CNN model, based on YOLOv3, was sensitive for detecting rib fractures on chest radiographs and showed great potential in the preliminary screening of rib fractures, which indicated that CNN can help reduce missed diagnoses and relieve radiologists’ workload. In this study, we developed and verified the performance of a novel CNN model for rib fracture detection by using radiography.

Automated deep-learning system in the assessment of MRI-visible prostate cancer: comparison of advanced zoomed diffusion-weighted imaging and conventional technique

BackgroundDeep-learning-based computer-aided diagnosis (DL-CAD) systems using MRI for prostate cancer (PCa) detection have demonstrated good performance. Nevertheless, DL-CAD systems are vulnerable to high heterogeneities in DWI, which can interfere with DL-CAD assessments and impair performance. This study aims to compare PCa detection of DL-CAD between zoomed-field-of-view echo-planar DWI (z-DWI) and full-field-of-view DWI (f-DWI) and find the risk factors affecting DL-CAD diagnostic efficiency.MethodsThis retrospective study enrolled 354 consecutive participants who underwent MRI including T2WI, f-DWI, and z-DWI because of clinically suspected PCa. A DL-CAD was used to compare the performance of f-DWI and z-DWI both on a patient level and lesion level. We used the area under the curve (AUC) of receiver operating characteristics analysis and alternative free-response receiver operating characteristics analysis to compare the performances of DL-CAD using f- DWI and z-DWI. The risk factors affecting the DL-CAD were analyzed using logistic regression analyses. P values less than 0.05 were considered statistically significant.ResultsDL-CAD with z-DWI had a significantly better overall accuracy than that with f-DWI both on patient level and lesion level (AUCpatient: 0.89 vs. 0.86; AUClesion: 0.86 vs. 0.76; P < .001). The contrast-to-noise ratio (CNR) of lesions in DWI was an independent risk factor of false positives (odds ratio [OR] = 1.12; P < .001). Rectal susceptibility artifacts, lesion diameter, and apparent diffusion coefficients (ADC) were independent risk factors of both false positives (ORrectal susceptibility artifact = 5.46; ORdiameter, = 1.12; ORADC = 0.998; all P < .001) and false negatives (ORrectal susceptibility artifact = 3.31; ORdiameter = 0.82; ORADC = 1.007; all P ≤ .03) of DL-CAD.ConclusionsZ-DWI has potential to improve the detection performance of a prostate MRI based DL-CAD.Trial registrationChiCTR, NO. ChiCTR2100041834. Registered 7 January 2021.

Added value of contrast enhancement boost images in routine multiphasic contrast-enhanced CT for the diagnosis of small (

PurposeTo investigate the added value of contrast enhancement boost (CE-boost) images in multiphasic contrast-enhanced CT (CE-CT) for diagnosing small (<20 mm) hypervascular hepatocellular carcinoma (HCC). Materials and methodsThis retrospective study included 69 patients (age, 74 ± 8 years; 52 men) with 70 hypervascular HCCs (<20 mm) who underwent multiphasic CE-CT (pre-contrast, late arterial phase [LAP], portal venous phase [PVP], and equilibrium phase). Two types of CE-boost images were generated by subtracting PVP from LAP (LA-PV) images and LAP from PVP (PV-LA) images to enhance the contrast effect of hepatic arterial and portal venous perfusion more selectively. Tumor-to-liver contrast-to-noise ratios (CNRs) in CE-boost images were compared with those in CE-CT images using the Wilcoxon signed-rank test. Two independent readers reviewed the imaging datasets: CE-CT alone and CE-CT with CE-boost images. The diagnostic performance of each dataset was compared using jackknife alternative free-response receiver operating characteristics (JAFROC-1). ResultsThe tumor-to-liver CNRs in the LA-PV (6.4 ± 3.0) and PV-LA (−3.3 ± 2.1) images were greater than those in the LAP (3.2 ± 1.7) and PVP images (−1.1 ± 1.4) (p <.001 for both). The reader-averaged figures of merit were 0.751 for CE-CT alone and 0.807 for CE-CT with CE-boost images (p <.001). Sensitivities increased by adding CE-boost images for both readers (p <.001 and = 0.03), while positive predictive values were equivalent (p >.99). ConclusionAdding CE-boost images to multiphasic CE-CT can improve the diagnostic accuracy and sensitivity for small hypervascular HCC by increasing the tumor-to-liver CNR.

Applicability of multidimensional convolutional neural networks on automated detection of diverse focal liver lesions in multiphase CT images.

To investigate the applicability of multidimensional convolutional neural networks (CNNs) together with multiphase contrast-enhanced CT images on automated detection of diverse focal liver lesions (FLLs). We trained detection models based on 2.5D and 3D CNN frameworks using 567 patients with 3892 FLLs and validated on a relatively large independent cohort of 1436 patients with 4723 lesions. The detection performance across different phases (arterial, portal venous [PV], and combined phases) was assessed for the 2.5D model. The lesions were divided into two groups with a cutoff size of 20mm, and further subdivided into four subgroups of<10, 10-20, 20-50, and ≥50mm, to verify the detection rates for lesions of different sizes for the 2.5D and 3D models. McNemar's test was used to compare the detection sensitivities among different methods. In addition, sensitivity with 95% confidence intervals and free-response receiver operating characteristics (FROC) curves were plotted for visualization of the detectability. In the 2.5D model, the detection rate of PV phase outperformed arterial phase, and a combination of the two phases further improved the performance over a single phase. The detection sensitivities in the arterial, PV, and combined phases were 0.737versus 0.802versus 0.832 for all lesions. The 3D model was superior to the 2.5D model for detecting benign lesions (0.896vs. 0.807, p<0.001), malignant lesions (0.940vs. 0.918, p=0.013), and all lesions (0.902vs. 0.832, p<0.001) regardless of size division. Particularly, the 3D model showed higher sensitivity than the 2.5D model in detecting lesions smaller than 20mm (0.868vs. 0.759, p<0.001). For lesions larger than 20mm, both the 3D and the 2.5D models achieved excellent detection performance. The proposed CNN detection model was demonstrated to adaptively learn the feature representations of diverse FLLs and generalize well to a large-scale validation dataset. The use of multiphase significantly improved the detectability of FLLs compared to single phase. 3D CNN framework showed an enhanced capability over the 2.5D in the detection of FLLs, particularly small lesions. The promising performance shows that the proposed CNN detection system could be a powerful clinical tool for the early detection of hepatic tumors.