Lesion Detection Performance Research Articles

Leveraging AI and patient metadata to develop a novel risk score for skin cancer detection

Melanoma of the skin is the 17th most common cancer worldwide. Early detection of suspicious skin lesions (melanoma) can increase 5-year survival rates by 20%. The 7-point checklist (7PCL) has been extensively used to suggest urgent referrals for patients with a possible melanoma. However, the 7PCL method only considers seven meta-features to calculate a risk score and is only relevant for patients with suspected melanoma. There are limited studies on the extensive use of patient metadata for the detection of all skin cancer subtypes. This study investigates artificial intelligence (AI) models that utilise patient metadata consisting of 23 attributes for suspicious skin lesion detection. We have identified a new set of most important risk factors, namely “C4C risk factors”, which is not just for melanoma, but for all types of skin cancer. The performance of the C4C risk factors for suspicious skin lesion detection is compared to that of the 7PCL and the Williams risk factors that predict the lifetime risk of melanoma. Our proposed AI framework ensembles five machine learning models and identifies seven new skin cancer risk factors: lesion pink, lesion size, lesion colour, lesion inflamed, lesion shape, lesion age, and natural hair colour, which achieved a sensitivity of 80.46±2.50%\\documentclass[12pt]{minimal} \\usepackage{amsmath} \\usepackage{wasysym} \\usepackage{amsfonts} \\usepackage{amssymb} \\usepackage{amsbsy} \\usepackage{mathrsfs} \\usepackage{upgreek} \\setlength{\\oddsidemargin}{-69pt} \\begin{document}$$80.46\\pm 2.50\\%$$\\end{document} and a specificity of 62.09±1.90%\\documentclass[12pt]{minimal} \\usepackage{amsmath} \\usepackage{wasysym} \\usepackage{amsfonts} \\usepackage{amssymb} \\usepackage{amsbsy} \\usepackage{mathrsfs} \\usepackage{upgreek} \\setlength{\\oddsidemargin}{-69pt} \\begin{document}$$62.09\\pm 1.90\\%$$\\end{document} in detecting suspicious skin lesions when evaluated using the metadata of 53,601 skin lesions collected from different skin cancer diagnostic clinics across the UK, significantly outperforming the 7PCL-based method (sensitivity 68.09±2.10%\\documentclass[12pt]{minimal} \\usepackage{amsmath} \\usepackage{wasysym} \\usepackage{amsfonts} \\usepackage{amssymb} \\usepackage{amsbsy} \\usepackage{mathrsfs} \\usepackage{upgreek} \\setlength{\\oddsidemargin}{-69pt} \\begin{document}$$68.09\\pm 2.10\\%$$\\end{document}, specificity 61.07±0.90%\\documentclass[12pt]{minimal} \\usepackage{amsmath} \\usepackage{wasysym} \\usepackage{amsfonts} \\usepackage{amssymb} \\usepackage{amsbsy} \\usepackage{mathrsfs} \\usepackage{upgreek} \\setlength{\\oddsidemargin}{-69pt} \\begin{document}$$61.07\\pm 0.90\\%$$\\end{document}) and the Williams risk factors (sensitivity 66.32±1.90%\\documentclass[12pt]{minimal} \\usepackage{amsmath} \\usepackage{wasysym} \\usepackage{amsfonts} \\usepackage{amssymb} \\usepackage{amsbsy} \\usepackage{mathrsfs} \\usepackage{upgreek} \\setlength{\\oddsidemargin}{-69pt} \\begin{document}$$66.32\\pm 1.90\\%$$\\end{document}, specificity 61.71±0.6%\\documentclass[12pt]{minimal} \\usepackage{amsmath} \\usepackage{wasysym} \\usepackage{amsfonts} \\usepackage{amssymb} \\usepackage{amsbsy} \\usepackage{mathrsfs} \\usepackage{upgreek} \\setlength{\\oddsidemargin}{-69pt} \\begin{document}$$61.71\\pm 0.6\\%$$\\end{document}). Furthermore, through weighting the seven new risk factors we came up with a new risk score, namely “C4C risk score”, which alone achieved a sensitivity of 76.09±1.20%\\documentclass[12pt]{minimal} \\usepackage{amsmath} \\usepackage{wasysym} \\usepackage{amsfonts} \\usepackage{amssymb} \\usepackage{amsbsy} \\usepackage{mathrsfs} \\usepackage{upgreek} \\setlength{\\oddsidemargin}{-69pt} \\begin{document}$$76.09\\pm 1.20\\%$$\\end{document} and a specificity of 61.71±0.50%\\documentclass[12pt]{minimal} \\usepackage{amsmath} \\usepackage{wasysym} \\usepackage{amsfonts} \\usepackage{amssymb} \\usepackage{amsbsy} \\usepackage{mathrsfs} \\usepackage{upgreek} \\setlength{\\oddsidemargin}{-69pt} \\begin{document}$$61.71\\pm 0.50\\%$$\\end{document}, significantly outperforming the 7PCL-based risk score (sensitivity 73.91±1.10%\\documentclass[12pt]{minimal} \\usepackage{amsmath} \\usepackage{wasysym} \\usepackage{amsfonts} \\usepackage{amssymb} \\usepackage{amsbsy} \\usepackage{mathrsfs} \\usepackage{upgreek} \\setlength{\\oddsidemargin}{-69pt} \\begin{document}$$73.91\\pm 1.10\\%$$\\end{document}, specificity 49.49±0.50%\\documentclass[12pt]{minimal} \\usepackage{amsmath} \\usepackage{wasysym} \\usepackage{amsfonts} \\usepackage{amssymb} \\usepackage{amsbsy} \\usepackage{mathrsfs} \\usepackage{upgreek} \\setlength{\\oddsidemargin}{-69pt} \\begin{document}$$49.49\\pm 0.50\\%$$\\end{document}) and the Williams risk score (sensitivity 60.68±1.30%\\documentclass[12pt]{minimal} \\usepackage{amsmath} \\usepackage{wasysym} \\usepackage{amsfonts} \\usepackage{amssymb} \\usepackage{amsbsy} \\usepackage{mathrsfs} \\usepackage{upgreek} \\setlength{\\oddsidemargin}{-69pt} \\begin{document}$$60.68\\pm 1.30\\%$$\\end{document}, specificity 60.87±0.80%\\documentclass[12pt]{minimal} \\usepackage{amsmath} \\usepackage{wasysym} \\usepackage{amsfonts} \\usepackage{amssymb} \\usepackage{amsbsy} \\usepackage{mathrsfs} \\usepackage{upgreek} \\setlength{\\oddsidemargin}{-69pt} \\begin{document}$$60.87\\pm 0.80\\%$$\\end{document}). Finally, fusing the C4C risk factors with the 7PCL and Williams risk factors achieved the best performance, with a sensitivity of 85.24±2.20%\\documentclass[12pt]{minimal} \\usepackage{amsmath} \\usepackage{wasysym} \\usepackage{amsfonts} \\usepackage{amssymb} \\usepackage{amsbsy} \\usepackage{mathrsfs} \\usepackage{upgreek} \\setlength{\\oddsidemargin}{-69pt} \\begin{document}$$85.24\\pm 2.20\\%$$\\end{document} and a specificity of 61.12±0.90%\\documentclass[12pt]{minimal} \\usepackage{amsmath} \\usepackage{wasysym} \\usepackage{amsfonts} \\usepackage{amssymb} \\usepackage{amsbsy} \\usepackage{mathrsfs} \\usepackage{upgreek} \\setlength{\\oddsidemargin}{-69pt} \\begin{document}$$61.12\\pm 0.90\\%$$\\end{document}. We believe that fusing these newly found risk factors and new risk score with image data will further boost the AI model performance for suspicious skin lesion detection. Hence, the new set of skin cancer risk factors has the potential to be used to modify current skin cancer referral guidelines for all skin cancer subtypes, including melanoma.

Leveraging Pretrained Transformers for Efficient Segmentation and Lesion Detection in Cone-Beam Computed Tomography Scans

IntroductionCone-beam computed tomography (CBCT) is widely used to detect jaw lesions, although CBCT interpretation is time-consuming and challenging. Artificial intelligence for CBCT segmentation may improve lesion detection accuracy. However, consistent automated lesion detection remains difficult, especially with limited training data. This study aimed to assess the applicability of pretrained transformer-based architectures for semantic segmentation of CBCT volumes when applied to periapical lesion detection. MethodsCBCT volumes (n = 138) were collected and annotated by expert clinicians using 5 labels – "lesion," "restorative material," "bone," "tooth structure," and "background." U-Net (convolutional neural network-based) and Swin-UNETR (transformer-based) models, pretrained (Swin-UNETR-PRETRAIN), and from scratch (Swin-UNETR-SCRATCH), were trained with subsets of the annotated CBCTs. These models were then evaluated for semantic segmentation performance using the Sørensen–Dice coefficient (DICE), lesion detection performance using sensitivity and specificity, and training sample size requirements by comparing models trained with 20, 40, 60, or 103 samples. ResultsTrained with 103 samples, Swin-UNETR-PRETRAIN achieved a DICE of 0.8512 for "lesion," 0.8282 for "restorative materials," 0.9178 for "bone," 0.9029 for "tooth structure," and 0.9901 for "background." “Lesion” DICE was statistically similar between Swin-UNETR-PRETRAIN trained with 103 and 60 images (P > .05), with the latter achieving 1.00 sensitivity and 0.94 specificity in lesion detection. With small training sets, Swin-UNETR-PRETRAIN outperformed Swin-UNETR-SCRATCH in DICE over all labels (P < .001 [n = 20], P < .001 [n = 40]), and U-Net in lesion detection specificity (P = .006 [n = 20], P = .031 [n = 40]). ConclusionsTransformer-based Swin-UNETR architectures allowed for excellent semantic segmentation and periapical lesion detection. Pretrained, it may provide an alternative with smaller training datasets compared to classic U-Net architectures.

A quantitative analysis of the improvement provided by comprehensive annotation on CT lesion detection using deep learning.

Data collected from hospitals are usually partially annotated by radiologists due to time constraints. Developing and evaluating deep learning models on these data may result in over or underestimation PURPOSE: We aimed to quantitatively investigate how the percentage of annotated lesions in CT images will influence the performance of universal lesion detection (ULD)algorithms. We trained a multi-view feature pyramid network with position-aware attention (MVP-Net) to perform ULD. Three versions of the DeepLesion dataset were created for training MVP-Net. Original DeepLesion Dataset (OriginalDL) is the publicly available, widely studied DeepLesion dataset that includes 32735 lesions in 4427 patients which were partially labeled during routine clinical practice. Enriched DeepLesion Dataset (EnrichedDL) is an enhanced dataset that features fully labeled at one or more time points for 4145 patients with 34317 lesions. UnionDL is the union of the OriginalDL and EnrichedDL with 54510 labeled lesions in 4427 patients. Each dataset was used separately to train MVP-Net, resulting in the following models: OriginalCNN (replicating the original result), EnrichedCNN (testing the effect of increased annotation), and UnionCNN (featuring the greatest number of annotations). Although the reported mean sensitivity of OriginalCNN was 84.3% using the OriginalDL testing set, the performance fell sharply when tested on the EnrichedDL testing set, yielding mean sensitivities of 56.1%, 66.0%, and 67.8% for OriginalCNN, EnrichedCNN, and UnionCNN, respectively. We also found that increasing the percentage of annotated lesions in the training set increased sensitivity, but the margin of increase in performance gradually diminished according to the powerlaw. We expanded and improved the existing DeepLesion dataset by annotating additional 21775 lesions, and we demonstrated that using fully labeled CT images avoided overestimation of MVP-Net's performance while increasing the algorithm's sensitivity, which may have a huge impact to the future CT lesion detection research. The annotated lesions are at https://github.com/ComputationalImageAnalysisLab/DeepLesionData.

Developing the Benchmark: Establishing a Gold Standard for the Evaluation of AI Caries Diagnostics.

Background/Objectives: The aim of this study was to establish a histology-based gold standard for the evaluation of artificial intelligence (AI)-based caries detection systems on proximal surfaces in bitewing images. Methods: Extracted human teeth were used to simulate intraoral situations, including caries-free teeth, teeth with artificially created defects and teeth with natural proximal caries. All 153 simulations were radiographed from seven angles, resulting in 1071 in vitro bitewing images. Histological examination of the carious lesion depth was performed twice by an expert. A total of thirty examiners analyzed all the radiographs for caries. Results: We generated in vitro bitewing images to evaluate the performance of AI-based carious lesion detection against a histological gold standard. All examiners achieved a sensitivity of 0.565, a Matthews correlation coefficient (MCC) of 0.578 and an area under the curve (AUC) of 76.1. The histology receiver operating characteristic (ROC) curve significantly outperformed the examiners' ROC curve (p < 0.001). All examiners distinguished induced defects from true caries in 54.6% of cases and correctly classified 99.8% of all teeth. Expert caries classification of the histological images showed a high level of agreement (intraclass correlation coefficient (ICC) = 0.993). Examiner performance varied with caries depth (p ≤ 0.008), except between E2 and E1 lesions (p = 1), while central beam eccentricity, gender, occupation and experience had no significant influence (all p ≥ 0.411). Conclusions: This study successfully established an unbiased dataset to evaluate AI-based caries detection on bitewing surfaces and compare it to human judgement, providing a standardized assessment for fair comparison between AI technologies and helping dental professionals to select reliable diagnostic tools.

RefineDet with Improved Attention Block for Chest X-ray Image Lesion Detection

In recent years, artificial intelligence technology has been widely used to assist radiologists in diagnosing and analyzing medical images. The use of artificial intelligence technology can well assist doctors in the localization of lesions. However, the mainstream target detection models at this stage are difficult to be practically applied in medical systems because of factors such as the use of large backbone networks and high input resolution, which leads to low model accuracy and high consumption of computational resources. In this paper, we propose a fast detection speed and high accuracy of the lung lesion detection network IAB- RefineDet. By improving the channel and spatial attention mechanisms and introducing the improved attention module into RefineDet, the lesion detection accuracy is dramatically improved without significantly increasing the number of parameters. We conduct extensive experiments on VinDr-CXR, the world's largest publicly available chest radiograph detection dataset, and comparative experiments with existing mainstream target detection models. The experimental results show that IAB-RefineDet achieves a mAP of 16.23%, and the lesion detection performance is significantly better than mainstream deep learning models.

Deep learning-based automated lesion segmentation on pediatric focal cortical dysplasia II preoperative MRI: a reliable approach

ObjectivesFocal cortical dysplasia (FCD) represents one of the most common causes of refractory epilepsy in children. Deep learning demonstrates great power in tissue discrimination by analyzing MRI data. A prediction model was built and verified using 3D full-resolution nnU-Net for automatic lesion detection and segmentation of children with FCD II.MethodsHigh-resolution brain MRI structure data from 65 patients, confirmed with FCD II by pathology, were retrospectively studied. Experienced neuroradiologists segmented and labeled the lesions as the ground truth. Also, we used 3D full-resolution nnU-Net to segment lesions automatically, generating detection maps. The algorithm was trained using fivefold cross-validation, with data partitioned into training (N = 200) and testing (N = 15). To evaluate performance, detection maps were compared to expert manual labels. The Dice-Sørensen coefficient (DSC) and sensitivity were used to assess the algorithm performance.ResultsThe 3D nnU-Net showed a good performance for FCD lesion detection at the voxel level, with a sensitivity of 0.73. The best segmentation model achieved a mean DSC score of 0.57 on the testing dataset.ConclusionThis pilot study confirmed that 3D full-resolution nnU-Net can automatically segment FCD lesions with reliable outcomes. This provides a novel approach to FCD lesion detection.Critical relevance statementOur fully automatic models could process the 3D T1-MPRAGE data and segment FCD II lesions with reliable outcomes.Key points• Simplified image processing promotes the DL model implemented in clinical practice.• The histopathological confirmed lesion masks enhance the clinical credibility of the AI model.• The voxel-level evaluation metrics benefit lesion detection and clinical decisions.Graphical

Learning Without Real Data Annotations to Detect Hepatic Lesions in PET Images.

Deep neural networks have been recently applied to lesion identification in fluorodeoxyglucose (FDG) positron emission tomography (PET) images, but they typically rely on a large amount of well-annotated data for model training. This is extremely difficult to achieve for neuroendocrine tumors (NETs), because of low incidence of NETs and expensive lesion annotation in PET images. The objective of this study is to design a novel, adaptable deep learning method, which uses no real lesion annotations but instead low-cost, list mode-simulated data, for hepatic lesion detection in real-world clinical NET PET images. We first propose a region-guided generative adversarial network (RG-GAN) for lesion-preserved image-to-image translation. Then, we design a specific data augmentation module for our list-mode simulated data and incorporate this module into the RG-GAN to improve model training. Finally, we combine the RG-GAN, the data augmentation module and a lesion detection neural network into a unified framework for joint-task learning to adaptatively identify lesions in real-world PET data. The proposed method outperforms recent state-of-the-art lesion detection methods in real clinical 68Ga-DOTATATE PET images, and produces very competitive performance with the target model that is trained with real lesion annotations. With RG-GAN modeling and specific data augmentation, we can obtain good lesion detection performance without using any real data annotations. This study introduces an adaptable deep learning method for hepatic lesion identification in NETs, which can significantly reduce human effort for data annotation and improve model generalizability for lesion detection with PET imaging.

Location-Aware Encoding for Lesion Detection in 68Ga-DOTATATE Positron Emission Tomography Images.

Lesion detection with positron emission tomography (PET) imaging is critical for tumor staging, treatment planning, and advancing novel therapies to improve patient outcomes, especially for neuroendocrine tumors (NETs). Current lesion detection methods often require manual cropping of regions/volumes of interest (ROIs/VOIs) a priori, or rely on multi-stage, cascaded models, or use multi-modality imaging to detect lesions in PET images. This leads to significant inefficiency, high variability and/or potential accumulative errors in lesion quantification. To tackle this issue, we propose a novel single-stage lesion detection method using only PET images. We design and incorporate a new, plug-and-play codebook learning module into a U-Net-like neural network and promote lesion location-specific feature learning at multiple scales. We explicitly regularize the codebook learning with direct supervision at the network's multi-level hidden layers and enforce the network to learn multi-scale discriminative features with respect to predicting lesion positions. The network automatically combines the predictions from the codebook learning module and other layers via a learnable fusion layer. We evaluate the proposed method on a real-world clinical 68Ga-DOTATATE PET image dataset, and our method produces significantly better lesion detection performance than recent state-of-the-art approaches. We present a novel deep learning method for single-stage lesion detection in PET imaging data, with no ROI/VOI cropping in advance, no multi-stage modeling and no multi-modality data. This study provides a new perspective for effective and efficient lesion identification in PET, potentially accelerating novel therapeutic regimen development for NETs and ultimately improving patient outcomes including survival.

Learned Full Waveform Inversion Incorporating Task Information for Ultrasound Computed Tomography.

Ultrasound computed tomography (USCT) is an emerging imaging modality that holds great promise for breast imaging. Full-waveform inversion (FWI)-based image reconstruction methods incorporate accurate wave physics to produce high spatial resolution quantitative images of speed of sound or other acoustic properties of the breast tissues from USCT measurement data. However, the high computational cost of FWI reconstruction represents a significant burden for its widespread application in a clinical setting. The research reported here investigates the use of a convolutional neural network (CNN) to learn a mapping from USCT waveform data to speed of sound estimates. The CNN was trained using a supervised approach with a task-informed loss function aiming at preserving features of the image that are relevant to the detection of lesions. A large set of anatomically and physiologically realistic numerical breast phantoms (NBPs) and corresponding simulated USCT measurements was employed during training. Once trained, the CNN can perform real-time FWI image reconstruction from USCT waveform data. The performance of the proposed method was assessed and compared against FWI using a hold-out sample of 41 NBPs and corresponding USCT data. Accuracy was measured using relative mean square error (RMSE), structural self-similarity index measure (SSIM), and lesion detection performance (DICE score). This numerical experiment demonstrates that a supervised learning model can achieve accuracy comparable to FWI in terms of RMSE and SSIM, and better performance in terms of task performance, while significantly reducing computational time.

A comparative study on deep feature selection methods for skin lesion classification

AbstractMelanoma, a widespread and hazardous form of cancer, has prompted researchers to prioritize dermoscopic image‐based algorithms for classifying skin lesions. Recently, there has been a growing trend in using pre‐trained convolutional neural networks for detecting skin lesions. However, the features extracted from these classifiers may include irrelevant elements, emphasizing the importance of implementing effective feature selection methods. Nevertheless, there has not been a comprehensive study on feature selection methods to enhance the performance of skin lesion detection to date. To identify the most efficient methods, a diverse set of feature selection techniques, including filter, wrapper, embedded, and dimensionality reduction, were applied to images from two well‐known datasets, namely ISIC 2017 and ISIC 2018. According to the results, models trained with features chosen by wrapper techniques outperformed those trained with features chosen by filter and embedded methods. Achieving an accuracy of 0.8333 and an F1‐Score of 0.8291 for the ISIC 2017 dataset, and an F1‐Score of 0.9324 and an accuracy of 0.9350 for the ISIC 2018 dataset, the classification using features obtained via the GWO feature selection technique performed the best.

CAPNet: Context attention pyramid network for computer-aided detection of microcalcification clusters in digital breast tomosynthesis

Background and objectiveComputer-aided detection (CADe) of microcalcification clusters (MCs) in digital breast tomosynthesis (DBT) is crucial in the early diagnosis of breast cancer. Although convolutional neural network (CNN)-based detection models have achieved excellent performance in medical lesion detection, they are subject to some limitations in MC detection: 1) Most existing models employ the feature pyramid network (FPN) for multi-scale object detection; however, the rough feature sharing between adjacent layers in the FPN may limit the detection ability for small and low-contrast MCs; and 2) the MCs region only accounts for a small part of the annotation box, so the features extracted indiscriminately within the whole box may easily be affected by the background. In this paper, we develop a novel CNN-based CADe method to alleviate the impacts of the above limitations for the accurate and rapid detection of MCs in DBT. MethodsThe proposed method has two parts: a novel context attention pyramid network (CAPNet) for intra-layer MC detection in two-dimensional (2D) slices and a three-dimensional (3D) aggregation procedure for aggregating 2D intra-layer MCs into a 3D result according to their connectivity in 3D space. The proposed CAPNet is based on an anchor-free and one-stage detection architecture and contains a context feature selection fusion (CFSF) module and a microcalcification response (MCR) branch. The CFSF module can efficiently enrich shallow layers’ features by the complementary selection of local context features, aiming to reduce the missed detection of small and low-contrast MCs. The MCR branch is a one-layer branch parallel to the classification branch, which can alleviate the influence of the background region within the annotation box on feature extraction and enhance the ability of the model to distinguish MCs from normal breast tissue. ResultsWe performed a comparison experiment on an in-house clinical dataset with 648 DBT volumes, and the proposed method achieved impressive performance with a sensitivity of 91.56 % at 1 false positive per DBT volume (FPs/volume) and 93.51 % at 2 FPs/volume, outperforming other representative detection models. ConclusionsThe experimental results indicate that the proposed method is effective in the detection of MCs in DBT. This method can provide objective, accurate, and quick diagnostic suggestions for radiologists, presenting potential clinical value for early breast cancer screening.

Deep Learning by Domain Transfer for Early Tumor Detection in the Urinary Bladder

Abstract Background: Bladder cancer (BCa) is the second most common genitourinary malignancy and has a mortality of 165,000 deaths p.a. The diagnosis of BCa is mostly carried out using cystoscopy - the visual examination of the urinary bladder with an endoscope. White light cystoscopy is currently considered as gold standard for the diagnosis. Nevertheless, especially flat, small or weakly textured lesions, are very difficult to detect and diagnose. Objective: With the advent of deep learning and already commercially available systems for the detection of adenomas in colonoscopy, it is investigated how such a system - for colonoscopy - performs if retrained and tested with cystoscopy images. Methods: A deep neural network with a YOLOv7-tiny architecture was pre-trained on 35,699 colonoscopy images (partially from Mannheim), yielding a precision = 0.92, sensitivity = 0.90, F1 = 0.91 on public colonoscopy data collections. Results: Testing this adenomadetection network with cystoscopy images from three sources (Ulm, Erlangen, Pforzheim), F1 scores in the range of 0.67 to 0.74 could be achieved. The network was then retrained with 12,066 cystoscopy images (from Mannheim), yielding improved F1 scores in the range of 0.78 to 0.85. Conclusion: It could be shown that a deep learning network for adenoma detection in colonoscopy is ad-hoc able to detect approximately 75% of the lesions in the urinary bladder in cystoscopy images, suggesting that these lesions have a similar appearance. After retraining the network with additional cystoscopy data, the performance for urinary lesion detection could be improved, indicating that a domain-shift with adequate additional data is feasible.

Pre-whitened matched filter and convolutional neural network based model observer performance for mass lesion detection in non-contrast breast CT.

Mathematical model observers have been shown to reasonably predict human observer performance and are useful when human observer studies are infeasible. Recently, convolutional neural networks (CNNs) have also been used as substitutes for human observers, and studies have shown their utility as an optimal observer. In this study, a CNN model observer is compared to the pre-whitened matched filter (PWMF) model observer in detecting simulated mass lesions inserted into 253 acquired breast computed tomography (bCT) images from patients imaged at our institution. To compare CNN and PWMF model observers for detecting signal-known-exactly (SKE) location-known-exactly (LKE) simulated lesions in bCT images with real anatomical backgrounds, and to use these model observers collectively to optimize parameters and understand trends in performance with breast CT. Spherical lesions with different diameters (1, 3, 5, 9mm) were mathematically inserted into reconstructed patient bCT image data sets to mimic 3D mass lesions in the breast. 2D images were generated by extracting the center slice along the axial dimension or by slice averaging across adjacent slices to model thicker sections (0.4, 1.2, 2.0, 6.0, 12.4, 20.4mm). The role of breast density was retrospectively studied using the range of breast densities intrinsic to the patient bCT data sets. In addition, mass lesions were mathematically inserted into Gaussian images matched to the mean and noise power spectrum of the bCT images to better understand the performance of the CNN in the context of a known ideal observer (the PWMF). The simulated Gaussian and bCT images were divided into training and testing data sets. Each training data set consisted of 91 600 images, and each testing data set consisted of 96 000 images. A CNN and PWMF was trained on the Gaussian training images, and a different CNN and PWMF was trained on the bCT training images. The trained model observers were tested, and receiver operating characteristic (ROC) curve analysis was used to evaluate detection performance. The area under the ROC curve (AUC) was the primary performance metric used to compare the model observers. In the Gaussian background, the CNN performed essentially identically to the PWMF across lesion sizes and section thicknesses. In the bCT background, the CNN outperformed the PWMF across lesion size, breast density, and most section thicknesses. These findings suggest that there are higher-order features in bCT images that are harnessed by the CNN observer but are inaccessible to the PWMF. The CNN performed equivalently to the ideal observer in Gaussian textures. In bCT background, the CNN captures more diagnostic information than the PWMF and may be a more pertinent observer when conducting optimal performance studies in breast CT images.

Application of a validated prostate MRI deep learning system to independent same-vendor multi-institutional data: demonstration of transferability

ObjectivesTo evaluate a fully automatic deep learning system to detect and segment clinically significant prostate cancer (csPCa) on same-vendor prostate MRI from two different institutions not contributing to training of the system.Materials and methodsIn this retrospective study, a previously bi-institutionally validated deep learning system (UNETM) was applied to bi-parametric prostate MRI data from one external institution (A), a PI-RADS distribution-matched internal cohort (B), and a csPCa stratified subset of single-institution external public challenge data (C). csPCa was defined as ISUP Grade Group ≥ 2 determined from combined targeted and extended systematic MRI/transrectal US-fusion biopsy. Performance of UNETM was evaluated by comparing ROC AUC and specificity at typical PI-RADS sensitivity levels. Lesion-level analysis between UNETM segmentations and radiologist-delineated segmentations was performed using Dice coefficient, free-response operating characteristic (FROC), and weighted alternative (waFROC). The influence of using different diffusion sequences was analyzed in cohort A.ResultsIn 250/250/140 exams in cohorts A/B/C, differences in ROC AUC were insignificant with 0.80 (95% CI: 0.74–0.85)/0.87 (95% CI: 0.83–0.92)/0.82 (95% CI: 0.75–0.89). At sensitivities of 95% and 90%, UNETM achieved specificity of 30%/50% in A, 44%/71% in B, and 43%/49% in C, respectively. Dice coefficient of UNETM and radiologist-delineated lesions was 0.36 in A and 0.49 in B. The waFROC AUC was 0.67 (95% CI: 0.60–0.83) in A and 0.7 (95% CI: 0.64–0.78) in B. UNETM performed marginally better on readout-segmented than on single-shot echo-planar-imaging.ConclusionFor same-vendor examinations, deep learning provided comparable discrimination of csPCa and non-csPCa lesions and examinations between local and two independent external data sets, demonstrating the applicability of the system to institutions not participating in model training.Clinical relevance statementA previously bi-institutionally validated fully automatic deep learning system maintained acceptable exam-level diagnostic performance in two independent external data sets, indicating the potential of deploying AI models without retraining or fine-tuning, and corroborating evidence that AI models extract a substantial amount of transferable domain knowledge about MRI-based prostate cancer assessment.Key Points• A previously bi-institutionally validated fully automatic deep learning system maintained acceptable exam-level diagnostic performance in two independent external data sets.• Lesion detection performance and segmentation congruence was similar on the institutional and an external data set, as measured by the weighted alternative FROC AUC and Dice coefficient.• Although the system generalized to two external institutions without re-training, achieving expected sensitivity and specificity levels using the deep learning system requires probability thresholds to be adjusted, underlining the importance of institution-specific calibration and quality control.

Automatic segmentation of prostate cancer metastases in PSMA PET/CT images using deep neural networks with weighted batch-wise dice loss

PurposeAutomatic and accurate segmentation of lesions in images of metastatic castration-resistant prostate cancer has the potential to enable personalized radiopharmaceutical therapy and advanced treatment response monitoring. The aim of this study is to develop a convolutional neural networks-based framework for fully-automated detection and segmentation of metastatic prostate cancer lesions in whole-body PET/CT images. Methods525 whole-body PET/CT images of patients with metastatic prostate cancer were available for the study, acquired with the [18F]DCFPyL radiotracer that targets prostate-specific membrane antigen (PSMA). U-Net (1)-based convolutional neural networks (CNNs) were trained to identify lesions on paired axial PET/CT slices. Baseline models were trained using batch-wise dice loss, as well as the proposed weighted batch-wise dice loss (wDice), and the lesion detection performance was quantified, with a particular emphasis on lesion size, intensity, and location. We used 418 images for model training, 30 for model validation, and 77 for model testing. In addition, we allowed our model to take n = 0,2, …, 12 neighboring axial slices to examine how incorporating greater amounts of 3D context influences model performance. We selected the optimal number of neighboring axial slices that maximized the detection rate on the 30 validation images, and trained five neural networks with different architectures. ResultsModel performance was evaluated using the detection rate, Dice similarity coefficient (DSC) and sensitivity. We found that the proposed wDice loss significantly improved the lesion detection rate, lesion-wise DSC and lesion-wise sensitivity compared to the baseline, with corresponding average increases of 0.07 (p-value = 0.01), 0.03 (p-value = 0.01) and 0.04 (p-value = 0.01), respectively. The inclusion of the first two neighboring axial slices in the input likewise increased the detection rate by 0.17, lesion-wise DSC by 0.05, and lesion-wise mean sensitivity by 0.16. However, there was a minimal effect from including more distant neighboring slices. We ultimately chose to use a number of neighboring slices equal to 2 and the wDice loss function to train our final model. To evaluate the model's performance, we trained three models using identical hyperparameters on three different data splits. The results showed that, on average, the model was able to detect 80% of all testing lesions, with a detection rate of 93% for lesions with maximum standardized uptake values (SUVmax) greater than 5.0. In addition, the average median lesion-wise DSC was 0.51 and 0.60 for all the lesions and lesions with SUVmax>5.0, respectively, on the testing set. Four additional neural networks with different architectures were trained, and they both yielded stronger performance of segmenting lesions whose SUVmax>5.0 compared to the rest of lesions. ConclusionOur results demonstrate that prostate cancer metastases in PSMA PET/CT images can be detected and segmented using CNNs. The segmentation performance strongly depends on the intensity, size, and the location of lesions, and can be improved by using specialized loss functions. Specifically, the models performed best in detection of lesions with SUVmax>5.0. Another challenge was to accurately segment lesions close to the bladder. Future work will focus on improving the detection of lesions with lower SUV values by designing custom loss functions that take into account the lesion intensity, using additional data augmentation techniques, and reducing the number of false lesions by developing methods to better separate signal from noise.

Multi-arm U-Net with dense input and skip connectivity for T2 lesion segmentation in clinical trials of multiple sclerosis

T2 lesion quantification plays a crucial role in monitoring disease progression and evaluating treatment response in multiple sclerosis (MS). We developed a 3D, multi-arm U-Net for T2 lesion segmentation, which was trained on a large, multicenter clinical trial dataset of relapsing MS. We investigated its generalization to other relapsing and primary progressive MS clinical trial datasets, and to an external dataset from the MICCAI 2016 MS lesion segmentation challenge. Additionally, we assessed the model’s ability to reproduce the separation of T2 lesion volumes between treatment and control arms; and the association of baseline T2 lesion volumes with clinical disability scores compared with manual lesion annotations. The trained model achieved a mean dice coefficient of ≥ 0.66 and a lesion detection sensitivity of ≥ 0.72 across the internal test datasets. On the external test dataset, the model achieved a mean dice coefficient of 0.62, which is comparable to 0.59 from the best model in the challenge, and a lesion detection sensitivity of 0.68. Lesion detection performance was reduced for smaller lesions (≤ 30 μL, 3–10 voxels). The model successfully maintained the separation of the longitudinal changes in T2 lesion volumes between the treatment and control arms. Such tools could facilitate semi-automated MS lesion quantification; and reduce rater burden in clinical trials.

Dual-Tracer Parathyroid Imaging Using Joint SPECT Reconstruction.

We assessed the lesion detection performance of the dual-tracer parathyroid SPECT imaging using the joint reconstruction method. Thirty-six noise realizations were created from SPECT projections collected from an in-house neck phantom to emulate 99mTc-pertechnetate/99mTc-sestamibi parathyroid SPECT datasets. Difference images representing parathyroid lesions were reconstructed using the subtraction and the joint methods whose corresponding optimal iteration was defined as the iteration which maximized the channelized Hotelling observer signal-to-noise ratio (CHO-SNR). The joint method whose initial estimate was derived from the subtraction method at optimal iteration (the joint-AltInt method) was also assessed. In a study of 36 patients, a human-observer lesion-detection study was performed using difference images from the three methods at optimal iteration and the subtraction method with four iterations. The area under the receiver operating characteristic curve (AUC) was calculated for each method. In the phantom study, both the joint-AltInt method and the joint method improved SNR compared to the subtraction method at their optimal iteration by 444% and 81%, respectively. In the patient study, the joint-AltInt method yielded the highest AUC of 0.73 as compared with 0.72, 0.71, and 0.64 from the joint method, the subtraction method at optimal iteration, and the subtraction method at four iterations. At a specificity of at least 0.70, the joint-AltInt method yielded significantly higher sensitivity than the other methods (0.60 vs 0.46, 042, and 0.42; p < 0.05). The joint reconstruction method yielded higher lesion detectability than the conventional method and holds promise for dual-tracer parathyroid SPECT imaging.

Influence of uncertainty estimation techniques on false-positive reduction in liver lesion detection

Deep learning techniques show success in detecting objects in medical images, but still suffer from false-positive predictions that may hinder accurate diagnosis. The estimated uncertainty of the neural network output has been used to flag incorrect predictions. We study the role played by features computed from neural network uncertainty estimates and shape-based features computed from binary predictions in reducing false positives in liver lesion detection by developing a classification-based post-processing step for different uncertainty estimation methods. We demonstrate an improvement in the lesion detection performance of the neural network (with respect to F1-score) for all uncertainty estimation methods on two datasets, comprising abdominal MR and CT images respectively. We show that features computed from neural network uncertainty estimates tend not to contribute much toward reducing false positives. Our results show that factors like class imbalance (true over false positive ratio) and shape-based features extracted from uncertainty maps play an important role in distinguishing false positive from true positive predictions.

Abbreviated Molecular Breast Imaging: Feasibility and Future Considerations.

Molecular breast imaging (MBI) is a supplemental screening modality consistently demonstrating incremental cancer detection over mammography alone; however, its lengthy duration may limit widespread utilization. The study purpose was to assess feasibility of an abbreviated MBI protocol, providing readers with mediolateral oblique (MLO) projections only and assessing performance in lesion detection and localization. Retrospective IRB-exempt blinded reader study administered to 5 fellowship-trained breast imaging radiologists. Independent reads performed for 124 screening MBI cases, half abnormal and half negative/normal. Readers determined whether an abnormality was present, side of abnormality, and location of abnormality (medial/lateral). Abnormal cases had confirmatory biopsy or surgical pathology; normal cases had imaging follow-up ensuring true negative results. Sensitivity, specificity, positive predictive value (PPV), and negative predictive value (NPV) were calculated to assess performance. A false negative result indicated that a reader failed to detect abnormal uptake; a false positive result indicated a reader incorrectly called an abnormality for a negative case. Tests for association included chi-square, Fisher-exact, and analysis of variance. Mean reader performance for detecting abnormal uptake: sensitivity 96.8%, specificity 98.7%, PPV 98.8%, and NPV 96.9%. Accuracy in localizing lesions to the medial or lateral breast was 100%. There were no associations in reader performance with reader experience, reader technique, lesion morphology, or lesion pathology. Median lesion size was 1.0 cm (range: 0.4-8.0 cm). All readers correctly identified 97.7% (42/43) of lesions with malignant or elevated risk pathology. An abbreviated MBI protocol (MLO images only) maintained high accuracy in lesion detection and localization.

Developing breast lesion detection algorithms for digital breast tomosynthesis: Leveraging false positive findings.

Due to the complex nature of digital breast tomosynthesis (DBT) in imaging techniques, reading times are longer than 2D mammograms. A robust computer-aided diagnosis system in DBT could help radiologists reduce their workload and reading times. The purpose of this study was to develop algorithms for detecting biopsy-proven breast lesions on DBT using multi-depth level convolutional models and leveraging non-biopsied samples. As biopsied positive samples in a lesion dataset are limited, we hypothesized that false positive (FP) findings by detection algorithms from non-biopsied benign lesions could improve detection algorithms by using them as data augmentation. We first extracted 2D slices from DBT volumes with biopsy-proven breast lesions (cancer and benign), with non-biopsied benign lesions (actionable), and for controls. Then, to provide lesion continuity along the z-direction, we combined a lesion slice with its immediate adjacent slices to synthesize 2.5-dimensional (2.5D) images of the lesion by assigning them into R, G, and B color channels. We used 224 biopsy-proven lesions from 39 cancer and 62 benign patients from a DBTex challenge dataset of 1000 scans. We included the 2.5D images of immediate neighboring slices from the lesion's center to increase the number of training samples. For lesion detection, we used the YOLOv5 algorithm as our base network. We trained a baseline algorithm (medium-depth level) using biopsied samples to detect actionable FPs in non-biopsied images. Afterward, we fine-tuned the baseline model on the augmented image set (actionable FPs added). For lesion inferencing, we processed the DBT volume slice-by-slice to estimate bounding boxes in each slice, and then combined them by connecting bounding boxes along the depth via volumetric morphological closing. We trained an additional model (large) with deeper-depth levels by repeating the above process. Finally, we developed an ensemble algorithm by combining the medium and large detection models. We used the free-response operating characteristic curve to evaluate our algorithms. We reported mean sensitivity per FPs per DBT volume only for biopsied views and sensitivity at 2-false positives per image (2FPI) for all views. However, due to the limited accessibility to the truth of the challenge validation and test datasets, we used sensitivity at 2FPI for statistical evaluation. For the DBTex independent validation set, the medium baseline model achieved a mean sensitivity of 0.627 FPs per DBT volume, and a sensitivity of 0.640 at 2FPI. After adding actionable FP lesions, the model had an improved 2FPI of 0.769 over the baseline (p-value=0.013). Our ensemble algorithm with multi-depth levels (medium + large) achieved a mean sensitivity of 0.815 FPs per DBT volume and an improved sensitivity at 2FPI of 0.80 over the baseline (p-value<0.001) on the validation set. Finally, our ensemble model achieved a mean sensitivity of 0.786 FPs per DBT volume and a sensitivity of 0.743 at 2FPI on the DBTex independent test set. Our results show that actionable FP findings hold useful information for lesion detection algorithms, and our ensemble detection model with multi-depth levels improves lesion detection performance.