Ground Truth Manual Segmentation Research Articles

Scan-rescan variability of AI-enabled cardiac functional assessment from cine CMR

Abstract Background/Introduction Cine cardiovascular magnetic resonance (CMR) is the gold-standard technique for the assessment of cardiac function and structure. Manual analysis of cine CMR to estimate structural and functional biomarkers is costly and time-consuming, motivating the use of artificial intelligence (AI) for automation. However, the scan-rescan repeatability of AI derived biomarkers remains unknown. Purpose Measure the repeatability of AI-based cine CMR biomarkers in scan-rescan data. Methods 92 scan-rescan short-axis cine CMR were acquired from volunteers on the same day (i.e. 184 in total) and annotated with ground truth (GT) manual segmentations of the left and right ventricle (LV, RV) blood pools, and LV myocardium at end-diastole (ED) and end-systole (ES). AI segmentations and subsequent biomarker estimates were produced using the AI-CMR-QC tool described in [1]. Segmentation performance was assessed with Dice score, pixelwise sensitivity and specificity, and Hausdorff distance. Scan-rescan differences in cardiac biomarkers for each subject were used to assess repeatability. Repeatability of LV ejection fraction (EF) based classification (LVEF<40 = abnormal, 40>LVEF>50 = mildly reduced, LVEF>50 = normal) [2] was also estimated from scan-rescan data. Results The segmentation model obtained high Dice scores (> 88% for LV and RV blood pools, >83% for LV myocardium). The summary of scan-rescan biomarker differences in Fig. 1(a) highlights that biomarker repeatability is generally better in GT segmentations, with lower differences between the two scans. While performance is satisfactory, a more in-depth analysis reveals that in some pairs of scans there are inconsistencies in the segmentations, leading to fluctuations in the calculation of blood pool volumes, which affects the repeatability of the computed biomarkers, as seen in Fig. 2(b). For instance, significant changes in LVEF between pairs of scans can arise from inconsistencies in the segmentation of the LV in basal slices, as shown in Fig. 2(a). This has consequences in the classification of subjects based on LVEF, as shown in Fig. 1(b). Only two subjects belonged to the mildly reduced LVEF class, however, when comparing predicted scan and rescan LVEF, both sensitivity and specificity decrease, suggesting inconsistency in the biomarker estimation. Conclusion(s) While we have demonstrated successful AI cardiac segmentation and biomarker estimation based on conventional performance metrics, additional refinement is necessary to enhance reliability and consistency. Further work is needed to obtain a better delineation of the valve plane, which would help addressing the inconsistencies at the LV basal slice. An automated segmentation and analysis pipeline for cine CMR is vital to support overworked clinicians, however, demonstrably repeatable performance, particularly when looking at therapy response, will have a significant impact on patient care and clinical outcomes.Repeatability of GT and AI biomarkersRepeatability of GT and AI segmentations

Reinforcement learning-based anatomical maps for pancreas subregion and duct segmentation.

The pancreas is a complex abdominal organ with many anatomical variations, and therefore automated pancreas segmentation from medical images is a challengingapplication. In this paper, we present a framework for segmenting individual pancreatic subregions and the pancreatic duct from three-dimensional (3D) computed tomography (CT) images. A multiagent reinforcement learning (RL) network was used to detect landmarks of the head, neck, body, and tail of the pancreas, and landmarks along the pancreatic duct in a selected target CT image. Using the landmark detection results, an atlas of pancreases was nonrigidly registered to the target image, resulting in anatomical probability maps for the pancreatic subregions and duct. The probability maps were augmented with multilabel 3D U-Net architectures to obtain the final segmentationresults. To evaluate the performance of our proposed framework, we computed the Dice similarity coefficient (DSC) between the predicted and ground truth manual segmentations on a database of 82 CT images with manually segmented pancreatic subregions and 37 CT images with manually segmented pancreatic ducts. For the four pancreatic subregions, the mean DSC improved from 0.38, 0.44, and 0.39 with standard 3D U-Net, Attention U-Net, and shifted windowing (Swin) U-Net architectures, to 0.51, 0.47, and 0.49, respectively, when utilizing the proposed RL-based framework. For the pancreatic duct, the RL-based framework achieved a mean DSC of 0.70, significantly outperforming the standard approaches and existing methods on different datasets. The resulting accuracy of the proposed RL-based segmentation framework demonstrates an improvement against segmentation with standard U-Netarchitectures.

Automated quantification of wrist bone marrow oedema, pre- and post-treatment, in early rheumatoid arthritis.

Bone inflammation (osteitis) in early RA (ERA) manifests as bone marrow oedema (BME) and precedes the development of bone erosion. In this prospective, single-centre study, we developed an automated post-processing pipeline for quantifying the severity of wrist BME on T2-weighted fat-suppressed MRI. A total of 80 ERA patients [mean age 54 years (s.d. 12), 62 females] were enrolled at baseline and 49 (40 females) after 1 year of treatment. For automated bone segmentation, a framework based on a convolutional neural network (nnU-Net) was trained and validated (5-fold cross-validation) for 15 wrist bone areas at baseline in 60 ERA patients. For BME quantification, BME was identified by Gaussian mixture model clustering and thresholding. BME proportion (%) and relative BME intensity within each bone area were compared with visual semi-quantitative assessment of the RA MRI score (RAMRIS). For automated wrist bone area segmentation, overall bone Sørensen-Dice similarity coefficient was 0.91 (s.d. 0.02) compared with ground truth manual segmentation. High correlation (Pearson correlation coefficient r = 0.928, P < 0.001) between visual RAMRIS BME and automated BME proportion assessment was found. The automated BME proportion decreased after treatment, correlating highly (r = 0.852, P < 0.001) with reduction in the RAMRIS BME score. The automated model developed had an excellent segmentation performance and reliable quantification of both the proportion and relative intensity of wrist BME in ERA patients, providing a more objective and efficient alternative to RAMRIS BME scoring.

Training and Comparison of nnU-Net and DeepMedic Methods for Autosegmentation of Pediatric Brain Tumors.

Tumor segmentation is essential in surgical and treatment planning and response assessment and monitoring in pediatric brain tumors, the leading cause of cancer-related death among children. However, manual segmentation is time-consuming and has high interoperator variability, underscoring the need for more efficient methods. After training, we compared 2 deep-learning-based 3D segmentation models, DeepMedic and nnU-Net, with pediatric-specific multi-institutional brain tumor data based on multiparametric MR images. Multiparametric preoperative MR imaging scans of 339 pediatric patients (n = 293 internal and n = 46 external cohorts) with a variety of tumor subtypes were preprocessed and manually segmented into 4 tumor subregions, ie, enhancing tumor, nonenhancing tumor, cystic components, and peritumoral edema. After training, performances of the 2 models on internal and external test sets were evaluated with reference to ground truth manual segmentations. Additionally, concordance was assessed by comparing the volume of the subregions as a percentage of the whole tumor between model predictions and ground truth segmentations using the Pearson or Spearman correlation coefficients and the Bland-Altman method. The mean Dice score for nnU-Net internal test set was 0.9 (SD, 0.07) (median, 0.94) for whole tumor; 0.77 (SD, 0.29) for enhancing tumor; 0.66 (SD, 0.32) for nonenhancing tumor; 0.71 (SD, 0.33) for cystic components, and 0.71 (SD, 0.40) for peritumoral edema, respectively. For DeepMedic, the mean Dice scores were 0.82 (SD, 0.16) for whole tumor; 0.66 (SD, 0.32) for enhancing tumor; 0.48 (SD, 0.27) for nonenhancing tumor; 0.48 (SD, 0.36) for cystic components, and 0.19 (SD, 0.33) for peritumoral edema, respectively. Dice scores were significantly higher for nnU-Net (P ≤ .01). Correlation coefficients for tumor subregion percentage volumes were higher (0.98 versus 0.91 for enhancing tumor, 0.97 versus 0.75 for nonenhancing tumor, 0.98 versus 0.80 for cystic components, 0.95 versus 0.33 for peritumoral edema in the internal test set). Bland-Altman plots were better for nnU-Net compared with DeepMedic. External validation of the trained nnU-Net model on the multi-institutional Brain Tumor Segmentation Challenge in Pediatrics (BraTS-PEDs) 2023 data set revealed high generalization capability in the segmentation of whole tumor, tumor core (a combination of enhancing tumor, nonenhancing tumor, and cystic components), and enhancing tumor with mean Dice scores of 0.87 (SD, 0.13) (median, 0.91), 0.83 (SD, 0.18) (median, 0.89), and 0.48 (SD, 0.38) (median, 0.58), respectively. The pediatric-specific data-trained nnU-Net model is superior to DeepMedic for whole tumor and subregion segmentation of pediatric brain tumors.

Auto-segmentation of Adult-Type Diffuse Gliomas: Comparison of Transfer Learning-Based Convolutional Neural Network Model vs. Radiologists

Segmentation of glioma is crucial for quantitative brain tumor assessment, to guide therapeutic research and clinical management, but very time-consuming. Fully automated tools for the segmentation of multi-sequence MRI are needed. We developed and pretrained a deep learning (DL) model using publicly available datasets A (n = 210) and B (n = 369) containing FLAIR, T2WI, and contrast-enhanced (CE)-T1WI. This was then fine-tuned with our institutional dataset (n = 197) containing ADC, T2WI, and CE-T1WI, manually annotated by radiologists, and split into training (n = 100) and testing (n = 97) sets. The Dice similarity coefficient (DSC) was used to compare model outputs and manual labels. A third independent radiologist assessed segmentation quality on a semi-quantitative 5-scale score. Differences in DSC between new and recurrent gliomas, and between uni or multifocal gliomas were analyzed using the Mann–Whitney test. Semi-quantitative analyses were compared using the chi-square test. We found that there was good agreement between segmentations from the fine-tuned DL model and ground truth manual segmentations (median DSC: 0.729, std-dev: 0.134). DSC was higher for newly diagnosed (0.807) than recurrent (0.698) (p < 0.001), and higher for unifocal (0.747) than multi-focal (0.613) cases (p = 0.001). Semi-quantitative scores of DL and manual segmentation were not significantly different (mean: 3.567 vs. 3.639; 93.8% vs. 97.9% scoring ≥ 3, p = 0.107). In conclusion, the proposed transfer learning DL performed similarly to human radiologists in glioma segmentation on both structural and ADC sequences. Further improvement in segmenting challenging postoperative and multifocal glioma cases is needed.

Abstract TMP84: Multi-Institutional Validation of an Automated Segmentation Tool for Intracerebral Hemorrhage (ICH) and Peri-Hematomal Edema (PHE) on Head CT

Objective: In patients with acute intracerebral hemorrhage (ICH), the volume of hemorrhage, and perihematomal edema (PHE) are representative of primary, and secondary brain injury, respectively. Automated quantification of ICH and PHE volumes from admission non-contrast head CT can facilitate evaluation of large stroke datasets, and expedite treatment triage when volume cutoffs are applied. We aimed to train and externally validate an automated model for segmentation and quantification of ICH and PHE volumes on non-contrast head CT scans. Methods: For training of the model, we used the data from multicentric ATACH-2 clinical trial with head CTs from eleven medical institutes in six countries. The model was then exported and tested on two external datasets from Yale and University of Berlin. We designed an automated pipeline for extracting brain window from 3D non-contrast head CT, skull stripping, resampling to homogenous voxel size, and segmentation. For segmentation we applied a deep learning model based on 3D full resolution nnUNet. We used the Dice Similarity Coefficient (DSC), Hausdorff Distance (HD), and Volume Similarity (VS) between automated segmentation and ground truth manual segmentation volumes to evaluate the models’ performance. Results: A total of 854 patients from the ATACH-2 trial (854 х2, baseline and follow-up CT scans) were used for training of the model in 5-fold cross-validation. In validation folds, the model achieved a mean DSC=0.90±0.10, HD =1.42±16.0mm, and VS=0.95±0.10 for ICH; and DSC=0.74±0.12, HD =3.0±16.5 mm, and VS=0.91±0.12 for PHE. In the externaltesting cohort from Yale (200 patients х2, baseline and follow-up CT scans), the model archieved median DSC=0.93, HD=0.98 mm, and VS=0.97 for ICH; and median DSC=0.69, HD=4.52 mm, and VS=0.84 for PHE. In the externaltesting cohort from Charité Hospital, University of Berlin (915 baseline CT scans), the model archived median DSC=0.87, HD=4 mm, and VS=0.90 for ICH; and median DSC=0.65, HD=5 mm, VS=0.85 for PHE. Conclusion: We trained, validated, and externally tested an end-to-end automated tool for segmentation of both ICH and PHE on non-contrast head CT. Such tool can facilitate treatment triage for trials when volume cutoffs are applied for enrollment.

The impact of data augmentation and transfer learning on the performance of deep learning models for the segmentation of the hip on 3D magnetic resonance images

Different pathologies of the hip are characterized by the abnormal shape of the bony structures of the joint, namely the femur and the acetabulum. Three-dimensional (3D) models of the hip can be used for diagnosis, biomechanical simulation, and planning of surgical treatments. These models can be generated by building 3D surfaces of the joint's structures segmented on magnetic resonance (MR) images. Deep learning can avoid time-consuming manual segmentations, but its performance depends on the amount and quality of the available training data. Data augmentation and transfer learning are two approaches used when there is only a limited number of datasets. In particular, data augmentation can be used to artificially increase the size and diversity of the training datasets, whereas transfer learning can be used to build the desired model on top of a model previously trained with similar data. This study investigates the effect of data augmentation and transfer learning on the performance of deep learning for the automatic segmentation of the femur and acetabulum on 3D MR images of patients diagnosed with femoroacetabular impingement. Transfer learning was applied starting from a model trained for the segmentation of the bony structures of the shoulder joint, which bears some resemblance to the hip joint. Our results suggest that data augmentation is more effective than transfer learning, yielding a Dice similarity coefficient compared to ground-truth manual segmentations of 0.84 and 0.89 for the acetabulum and femur, respectively, whereas the Dice coefficient was 0.78 and 0.88 for the model based on transfer learning. The Accuracy for the two anatomical regions was 0.95 and 0.97 when using data augmentation, and 0.87 and 0.96 when using transfer learning. Data augmentation can improve the performance of deep learning models by increasing the diversity of the training dataset and making the models more robust to noise and variations in image quality. The proposed segmentation model could be combined with radiomic analysis for the automatic evaluation of hip pathologies.

A Deep Learning Pipeline for Assessing Ventricular Volumes from a Cardiac MRI Registry of Patients with Single Ventricle Physiology.

Purpose To develop an end-to-end deep learning (DL) pipeline for automated ventricular segmentation of cardiac MRI data from a multicenter registry of patients with Fontan circulation (Fontan Outcomes Registry Using CMR Examinations [FORCE]). Materials and Methods This retrospective study used 250 cardiac MRI examinations (November 2007-December 2022) from 13 institutions for training, validation, and testing. The pipeline contained three DL models: a classifier to identify short-axis cine stacks and two U-Net 3+ models for image cropping and segmentation. The automated segmentations were evaluated on the test set (n = 50) by using the Dice score. Volumetric and functional metrics derived from DL and ground truth manual segmentations were compared using Bland-Altman and intraclass correlation analysis. The pipeline was further qualitatively evaluated on 475 unseen examinations. Results There were acceptable limits of agreement (LOA) and minimal biases between the ground truth and DL end-diastolic volume (EDV) (bias: -0.6 mL/m2, LOA: -20.6 to 19.5 mL/m2) and end-systolic volume (ESV) (bias: -1.1 mL/m2, LOA: -18.1 to 15.9 mL/m2), with high intraclass correlation coefficients (ICCs > 0.97) and Dice scores (EDV, 0.91 and ESV, 0.86). There was moderate agreement for ventricular mass (bias: -1.9 g/m2, LOA: -17.3 to 13.5 g/m2) and an ICC of 0.94. There was also acceptable agreement for stroke volume (bias: 0.6 mL/m2, LOA: -17.2 to 18.3 mL/m2) and ejection fraction (bias: 0.6%, LOA: -12.2% to 13.4%), with high ICCs (>0.81). The pipeline achieved satisfactory segmentation in 68% of the 475 unseen examinations, while 26% needed minor adjustments, 5% needed major adjustments, and in 0.4%, the cropping model failed. Conclusion The DL pipeline can provide fast standardized segmentation for patients with single ventricle physiology across multiple centers. This pipeline can be applied to all cardiac MRI examinations in the FORCE registry. Keywords: Cardiac, Adults and Pediatrics, MR Imaging, Congenital, Volume Analysis, Segmentation, Quantification Supplemental material is available for this article. © RSNA, 2023.

An active learning approach to train a deep learning algorithm for tumor segmentation from brain MR images

PurposeThis study focuses on assessing the performance of active learning techniques to train a brain MRI glioma segmentation model.MethodsThe publicly available training dataset provided for the 2021 RSNA-ASNR-MICCAI Brain Tumor Segmentation (BraTS) Challenge was used in this study, consisting of 1251 multi-institutional, multi-parametric MR images. Post-contrast T1, T2, and T2 FLAIR images as well as ground truth manual segmentation were used as input for the model. The data were split into a training set of 1151 cases and testing set of 100 cases, with the testing set remaining constant throughout. Deep convolutional neural network segmentation models were trained using the NiftyNet platform. To test the viability of active learning in training a segmentation model, an initial reference model was trained using all 1151 training cases followed by two additional models using only 575 cases and 100 cases. The resulting predicted segmentations of these two additional models on the remaining training cases were then addended to the training dataset for additional training.ResultsIt was demonstrated that an active learning approach for manual segmentation can lead to comparable model performance for segmentation of brain gliomas (0.906 reference Dice score vs 0.868 active learning Dice score) while only requiring manual annotation for 28.6% of the data.ConclusionThe active learning approach when applied to model training can drastically reduce the time and labor spent on preparation of ground truth training data.Critical relevance statementActive learning concepts were applied to a deep learning-assisted segmentation of brain gliomas from MR images to assess their viability in reducing the required amount of manually annotated ground truth data in model training.Key points• This study focuses on assessing the performance of active learning techniques to train a brain MRI glioma segmentation model.• The active learning approach for manual segmentation can lead to comparable model performance for segmentation of brain gliomas.• Active learning when applied to model training can drastically reduce the time and labor spent on preparation of ground truth training data.Graphical

Neural network for autonomous segmentation and volumetric assessment of clot and edema in acute and subacute intracerebral hemorrhages

IntroductionMinimally-invasive surgical techniques for intracerebral hemorrhage (ICH) evacuation use imaging to guide the suction, lysing and/or drainage from the hemorrhage site via various designs. A previous international surgical study has shown that reduction of hematoma volume below 15 ml is indicative of improved long term patient outcomes. The study noted a need for tools to periodically visualize remaining clot during intervention to increase the likelihood of evacuating sufficient clot volumes without endangering rebleeds. Robust segmentation of MRI could guide surgeons and radiologists regarding remaining regions and approaches for prudent evacuation. We thus propose a Convolutional Neural Network (CNN) to identify and autonomously segment clot and peripheral edema in MR images of the brain and generate an estimate of the remaining clot volume. Materials and methodsWe used a retrospective, locally-acquired dataset of ICH patient scans taken on 3 T MRI scanners. Three sets of ground truth manual segmentations were independently generated by two imaging scientists and one radiology fellow. Evaluation of clot age was determined based on relative contrast of hemorrhage components and reviewed by a neurosurgeon. Model accuracy was determined by pixel-wise Dice coefficient (DC) calculations between each ground truth manual segmentation and the machine-derived autonomous segmentations. ResultsThe model produced autonomous segmentations of clot core with an average DC of 0.75 ± 0.21 relative to manual segmentations of the same scans. For edema, it produced segmentations with an average DC of 0.68 ± 0.16 relative to manual. From these pixel-wise segmentations, clot volume can be calculated. Model-produced segmentations underestimated clot volumes by an average of 17% relative to ground-truth. ConclusionThe machine learning models were able to identify and segment volumes of ICH components swiftly and accurately.

A Deep Learning Method for Quantification of Femoral Head Necrosis Based on Routine Hip MRI for Improved Surgical Decision Making.

(1) Background: To evaluate the performance of a deep learning model to automatically segment femoral head necrosis (FHN) based on a standard 2D MRI sequence compared to manual segmentations for 3D quantification of FHN. (2) Methods: Twenty-six patients (thirty hips) with avascular necrosis underwent preoperative MR arthrography including a coronal 2D PD-w sequence and a 3D T1 VIBE sequence. Manual ground truth segmentations of the necrotic and unaffected bone were then performed by an expert reader to train a self-configuring nnU-Net model. Testing of the network performance was performed using a 5-fold cross-validation and Dice coefficients were calculated. In addition, performance across the three segmentations were compared using six parameters: volume of necrosis, volume of unaffected bone, percent of necrotic bone volume, surface of necrotic bone, unaffected femoral head surface, and percent of necrotic femoral head surface area. (3) Results: Comparison between the manual 3D and manual 2D segmentations as well as 2D with the automatic model yielded significant, strong correlations (Rp > 0.9) across all six parameters of necrosis. Dice coefficients between manual- and automated 2D segmentations of necrotic- and unaffected bone were 75 ± 15% and 91 ± 5%, respectively. None of the six parameters of FHN differed between the manual and automated 2D segmentations and showed strong correlations (Rp > 0.9). Necrotic volume and surface area showed significant differences (all p < 0.05) between early and advanced ARCO grading as opposed to the modified Kerboul angle, which was comparable between both groups (p > 0.05). (4) Conclusions: Our deep learning model to automatically segment femoral necrosis based on a routine hip MRI was highly accurate. Coupled with improved quantification for volume and surface area, as opposed to 2D angles, staging and course of treatment can become better tailored to patients with varying degrees of AVN.

Automated segmentation of the healed anterior cruciate ligament from T2 * relaxometry MRI scans.

Collagen organization of the anterior cruciate ligament (ACL) can be evaluated using T2 * relaxometry. However, T2 * mapping requires manual image segmentation, which is a time-consuming process and prone to inter- and intra- segmenter variability. Automating segmentation would address these challenges. A model previously trained using Constructive Interference in Steady State (CISS) scans was applied to T2 * segmentation via transfer learning. It was hypothesized that there would be no significant differences in the model's segmentation performance between T2 * and CISS, structural measures versus ground truth manual segmentation, and reliability versus independent and retest manual segmentation. Transfer learning was conducted using 54 T2 * scans of the ACL. Segmentation performance was assessed with Dice coefficient, precision, and sensitivity, and structurally with T2 * value, volume, subvolume proportions, and cross-sectional area. Model performance relative to independent manual segmentation and repeated segmentation by the ground truth segmenter (retest) were evaluated on a random subset. Segmentation performance was analyzed with Mann-Whitney U tests, structural measures with Wilcoxon signed-rank tests, and performance relative to manual segmentation with repeated-measures analysis of variance/Tukey tests (α = 0.05). T2 * segmentation performance was not significantly different from CISS on all measures (p > 0.35). No significant differences were detected in structural measures (p > 0.50). Automatic segmentation performed as well as the retest on all segmentation measures, whereas independent segmentations were lower than retest and/or automatic segmentation (p < 0.023). Structural measures were not significantly different between segmenters. The automatic segmentation model performed as well on the T2 * sequence as on CISS and outperformed independent manual segmentation while performing as well as retest segmentation.

Automatic quantification of white matter hyperintensities on T2-weighted fluid attenuated inversion recovery magnetic resonance imaging

White matter hyperintensities (WMH) are areas of increased signal visualized on T2-weighted fluid attenuated inversion recovery (FLAIR) brain magnetic resonance imaging (MRI) sequences. They are typically attributed to small vessel cerebrovascular disease in the context of aging. Among older adults, WMH are associated with risk of cognitive decline and dementia, stroke, and various other health outcomes. There has been increasing interest in incorporating quantitative WMH measurement as outcomes in clinical trials, observational research, and clinical settings. Here, we present a novel, fully automated, unsupervised detection algorithm for WMH segmentation and quantification. The algorithm uses a robust preprocessing pipeline, including brain extraction and a sample-specific mask that incorporates spatial information for automatic false positive reduction, and a half Gaussian mixture model (HGMM). The method was evaluated in 24 participants with varying degrees of WMH (4.9–78.6 cm3) from a community-based study of aging and dementia with dice coefficient, sensitivity, specificity, correlation, and bias relative to the ground truth manual segmentation approach performed by two expert raters. Results were compared with those derived from commonly used available WMH segmentation packages, including SPM lesion probability algorithm (LPA), SPM lesion growing algorithm (LGA), and Brain Intensity AbNormality Classification Algorithm (BIANCA). The HGMM algorithm derived WMH values that had a dice score of 0.87, sensitivity of 0.89, and specificity of 0.99 compared to ground truth. White matter hyperintensity volumes derived with HGMM were strongly correlated with ground truth values (r = 0.97, p = 3.9e-16), with no observable bias (−1.1 [−2.6, 0.44], p-value = 0.16). Our novel algorithm uniquely uses a robust preprocessing pipeline and a half-Gaussian mixture model to segment WMH with high agreement with ground truth for large scale studies of brain aging.

Fully automated deep learning system for detecting sarcopenia on brain MRI in glioblastoma

Abstract Aims Glioblastoma multiforme (GBM) is an aggressive brain malignancy. Performance status is an important prognostic factor but is subjectively evaluated, resulting in inaccuracy. Objective markers of frailty/physical condition, such as measures of skeletal muscle mass can be evaluated on cross-sectional imaging and is associated with cancer survival. In GBM, temporalis muscle has been identified as a skeletal muscle mass surrogate and a prognostic factor. However, current manual muscle quantification is time consuming, limiting clinical adoption. We previously developed a deep learning system for automated temporalis muscle quantification, with high accuracy (Dice coefficient 0.912), and showed muscle cross-sectional area is independently significantly associated with survival in GBM (HR 0.380). However, it required manual selection of the temporalis muscle-containing MRI slice. Thus, in this work we aimed to develop a fully automatic deep-learning system, using the eyeball as an anatomic landmark for automatic slice selection, to quantify temporalis and validate on independent datasets. Method 3D brain MRI scans were obtained from four datasets: our in-house glioblastoma patient dataset, TCGA-GBM, IVY-GAP and REMBRANDT. Manual eyeball and temporalis segmentations were performed on 2D MRI images by two experienced readers. Two neural networks (2D U-Nets) were trained, one to automatically segment the eyeball and the other to segment the temporalis muscle on 2D MRI images using Dice loss function. The cross sectional area of eyeball segmentations were quantified and thresholded, to select the superior orbital MRI slice from each scan. This slice underwent temporalis segmentation, whose cross sectional area was then quantified. Accuracy of automatically predicted eyeball and temporalis segmentations were compared to manual ground truth segmentations on metrics of Dice coefficient, precision, recall and Hausdorff distance. Accuracy of MRI slice selection (by the eyeball segmentation model) for temporalis segmentation was determined by comparing automatically selected slices to slices selected manually by a trained neuro-oncologist. Results 398 images from 185 patients and 366 images from 145 patients were used for the eyeball and temporalis segmentation models, respectively. 61 independent TCGA-GBM scans formed a validation cohort to assess the performance of the full pipeline. The model achieved high accuracy in eyeball segmentation, with test set Dice coefficient of 0.9029 ± 0.0894, precision of 0.8842 ± 0.0992, recall of 0.9297 ± 0.6020 and Hausdorff distance of 2.8847 ± 0.6020. High segmentation accuracy was also achieved by the temporalis segmentation model, with Dice coefficient of 0.8968 ± 0.0375, precision of 0.8877 ± 0.0679, recall of 0.9118 ± 0.0505 and Hausdorff distance of 1.8232 ± 0.3263 in the test set. 96.1% of automatically selected slices for temporalis segmentation were within 2 slices of the manually selected slice. Conclusion Temporalis muscle cross-sectional area can be rapidly and accurately assessed from 3D MRI brain scans using a deep learning-based system in a fully automated pipeline. Combined with our and others’ previous results that demonstrate the prognostic significance of temporalis cross-sectional area and muscle width, our findings suggest a role for deep learning in muscle mass and sarcopenia screening in GBM, with the potential to add significant value to routine imaging. Possible clinical applications include risk profiling, treatment stratification and informing interventions for muscle preservation. Further work will be to validate the prognostic value of temporalis muscle cross sectional area measurements generated by our fully automatic deep learning system in the multiple in-house and external datasets.

Weakly supervised segmentation of tumor lesions in PET-CT hybrid imaging.

.Purpose: We introduce and evaluate deep learning methods for weakly supervised segmentation of tumor lesions in whole-body fluorodeoxyglucose-positron emission tomography (FDG-PET) based solely on binary global labels (“tumor” versus “no tumor”).Approach: We propose a three-step approach based on (i) a deep learning framework for image classification, (ii) subsequent generation of class activation maps (CAMs) using different CAM methods (CAM, GradCAM, GradCAM++, ScoreCAM), and (iii) final tumor segmentation based on the aforementioned CAMs. A VGG-based classification neural network was trained to distinguish between PET image slices with and without FDG-avid tumor lesions. Subsequently, the CAMs of this network were used to identify the tumor regions within images. This proposed framework was applied to FDG-PET/CT data of 453 oncological patients with available manually generated ground-truth segmentations. Quantitative segmentation performance was assessed for the different CAM approaches and compared with the manual ground truth segmentation and with supervised segmentation methods. In addition, further biomarkers (MTV and TLG) were extracted from the segmentation masks.Results: A weakly supervised segmentation of tumor lesions was feasible with satisfactory performance [best median Dice score 0.47, interquartile range (IQR) 0.35] compared with a fully supervised U-Net model (median Dice score 0.72, IQR 0.36) and a simple threshold based segmentation (Dice score 0.29, IQR 0.28). CAM, GradCAM++, and ScoreCAM yielded similar results. However, GradCAM led to inferior results (median Dice score: 0.12, IQR 0.21) and was likely to ignore multiple instances within a given slice. CAM, GradCAM++, and ScoreCAM yielded accurate estimates of metabolic tumor volume (MTV) and tumor lesion glycolysis. Again, worse results were observed for GradCAM.Conclusions: This work demonstrated the feasibility of weakly supervised segmentation of tumor lesions and accurate estimation of derived metrics such as MTV and tumor lesion glycolysis.

Automated Registration-Based Temporal Bone Computed Tomography Segmentation for Applications in Neurotologic Surgery.

This study investigates the accuracy of an automated method to rapidly segment relevant temporal bone anatomy from cone beam computed tomography (CT) images. Implementation of this segmentation pipeline has potential to improve surgical safety and decrease operative time by augmenting preoperative planning and interfacing with image-guided robotic surgical systems. Descriptive study of predicted segmentations. Academic institution. We have developed a computational pipeline based on the symmetric normalization registration method that predicts segmentations of anatomic structures in temporal bone CT scans using a labeled atlas. To evaluate accuracy, we created a data set by manually labeling relevant anatomic structures (eg, ossicles, labyrinth, facial nerve, external auditory canal, dura) for 16 deidentified high-resolution cone beam temporal bone CT images. Automated segmentations from this pipeline were compared against ground-truth manual segmentations by using modified Hausdorff distances and Dice scores. Runtimes were documented to determine the computational requirements of this method. Modified Hausdorff distances and Dice scores between predicted and ground-truth labels were as follows: malleus (0.100 ± 0.054 mm; Dice, 0.827 ± 0.068), incus (0.100 ± 0.033 mm; Dice, 0.837 ± 0.068), stapes (0.157 ± 0.048 mm; Dice, 0.358 ± 0.100), labyrinth (0.169 ± 0.100 mm; Dice, 0.838 ± 0.060), and facial nerve (0.522 ± 0.278 mm; Dice, 0.567 ± 0.130). A quad-core 16GB RAM workstation completed this segmentation pipeline in 10 minutes. We demonstrated submillimeter accuracy for automated segmentation of temporal bone anatomy when compared against hand-segmented ground truth using our template registration pipeline. This method is not dependent on the training data volume that plagues many complex deep learning models. Favorable runtime and low computational requirements underscore this method's translational potential.

Deep Learning-Based Retinal Nerve Fiber Layer Thickness Measurement of Murine Eyes.

PurposeTo design a robust and automated estimation method for measuring the retinal nerve fiber layer (RNFL) thickness using spectral domain optical coherence tomography (SD-OCT).MethodsWe developed a deep learning–based image segmentation network for automated segmentation of the RNFL in SD-OCT B-scans of mouse eyes. In total, 5500 SD-OCT B-scans (5200 B-scans were used as training data with the remaining 300 B-scans used as testing data) were used to develop this segmentation network. Postprocessing operations were then applied on the segmentation results to fill any discontinuities or remove any speckles in the RNFL. Subsequently, a three-dimensional retina thickness map was generated by z-stacking 100 segmentation processed thickness B-scan images together. Finally, the average absolute difference between algorithm predicted RNFL thickness compared to the ground truth manual human segmentation was calculated.ResultsThe proposed method achieves an average dice similarity coefficient of 0.929 in the SD-OCT segmentation task and an average absolute difference of 0.0009 mm in thickness estimation task on the basis of the testing dataset. We also evaluated our segmentation algorithm on another biological dataset with SD-OCT volumes for RNFL thickness after the optic nerve crush injury. Results were shown to be comparable between the predicted and manually measured retina thickness values.ConclusionsExperimental results demonstrate that our automated segmentation algorithm reliably predicts the RNFL thickness in SD-OCT volumes of mouse eyes compared to laborious and more subjective manual SD-OCT RNFL segmentation.Translational RelevanceAutomated segmentation using a deep learning–based algorithm for murine eye OCT effectively and rapidly produced nerve fiber layer thicknesses comparable to manual segmentation.

Abstract 2600: Deep-LIBRA: An artificial intelligence approach for fully-automated assessment of breast density in digital mammography

Abstract Accurate estimation of mammographic breast density could aid in augmenting breast cancer risk assessment for women undergoing breast screening with full-field digital mammography (FFDM). Breast density can be estimated from FFDM and is most commonly assessed in the clinic by visual grading into one of the four categories defined by the American College of Radiology BI-RADS. However, BI-RADS density assessment is highly subjective and does not provide a quantitative, continuous measure of percent density (PD), which would allow for more refined risk stratification and assessment of density changes. Here, we introduce Deep-LIBRA, an artificial intelligence (AI) tool for fully-automated assessment of breast PD from FFDM images. Two key modules form the core of Deep-LIBRA: 1) an implementation of a modified U-Net architecture for breast segmentation and 2) a radiomic machine learning module that performs PD estimation within the segmented breast region. To develop and validate Deep-LIBRA, raw (i.e., "For Processing") FFDM images (Selenia Dimensions, Hologic Inc.) acquired at two breast cancer screening practices were retrospectively analyzed. For the breast segmentation module, we used a total of 12,100 FFDM studies from 2,200 individual women and a 90%-10% split-sample training-validation approach, using the Dice coefficient to evaluate the accuracy of Deep-LIBRA versus ground-truth manual breast segmentation. For the PD estimation module we used a total of 3,304 FFDM images from 1,652 individual women; manual PD scores obtained with the widely used Cumulus software were used as the "gold standard" in a three-fold cross-validation setting to assess the accuracy of Deep-LIBRA in PD estimation. PD estimates from Deep-LIBRA were also compared with breast density estimates from the commercially available Volpara software. Breast segmentation had a Dice coefficient of 95.31% when compared to ground-truth manual breast segmentation in the validation set. Deep-LIBRA average differences from ground-truth PD scores in the three cross-validation folds were 4.91%, 4.65%, and 4.22%, while Volpara had corresponding average differences of 6.20%, 6.01%, and 5.94%. Deep-LIBRA PD scores were also significantly different from Volpara PD (t-test p-value &amp;lt; 0.001 in all three folds). Preliminary evaluation results show that Deep-LIBRA is a promising AI approach for accurately assessing PD from FFDM images. Citation Format: Omid Haji Maghsoudi, Scott Christopher, Aimilia Gastounioti, Lauren Pantalone, Fang-Fang Wu, Eric A. Cohen, Winham Stacey, Emily F. Conant, Celine Vachon, Despina Kontos. Deep-LIBRA: An artificial intelligence approach for fully-automated assessment of breast density in digital mammography [abstract]. In: Proceedings of the American Association for Cancer Research Annual Meeting 2021; 2021 Apr 10-15 and May 17-21. Philadelphia (PA): AACR; Cancer Res 2021;81(13_Suppl):Abstract nr 2600.

A deep learning-based and fully automated pipeline for thoracic aorta geometric analysis and TEVAR planning from computed tomography

Abstract Funding Acknowledgements Type of funding sources: Public grant(s) – National budget only. Main funding source(s): Ministry of Publich Health - Ricerca Corrente Introduction Thoracic endovascular aortic repair (TEVAR) represents a well-established alternative to open repair in selected patients. Its preoperative feasibility assessment and planning requires a computational tomography (CT)-based analysis of the geometric aortic features to identify an adequate proximal and distal landing zone (LZ) for endograft deployment. Yet, controversies persist on the definition and methods of measurement of specific geometric features of the LZs, including angulation and tortuosity, which are associated with an increased risk of postoperative endograft failure. In this respect, the development of a preoperative image processing method that provides an automatic and highly reproducible 3D identification of critical geometric features and specific anatomical landmarks, thus reducing the time and uncertainties related to manual segmentation, remains a largely unmet clinical need. In this study, we developed and applied a fully automated pipeline embedding a convolutional neural network (CNN), which feeds on 3D CT images to automatically segment the thoracic aorta, recognize the relevant anatomical landmarks and LZs, and quantifies the geometry of the aortic arch in each proximal LZ s (i.e. 0 to 3). Methods Ninety CT scans of healthy aortas were retrieved, being the study conceived as a proof of concept analysis. The thoracic aorta was manually segmented by five independent and expert operators. 72 scans with the corresponding ground truth segmentations were randomly selected and used to train the CNN, which was based on a 3D U-Net architecture. The other 18 scans were used to test the CNN-based segmentations. The fully automated pipeline was obtained by integrating the CNN, 3D geometry skeletonization, and processing of the aortic centerline and wall via computational geometry (Figure). The resulting metrics included aortic arch centerline radius of curvature, proximal landing zones (PLZs) maximum diameters, angulation and tortuosity calculated according to previously published work. These parameters were statistically analyzed to compare standard arches vs. arches with a common origin of the innominate and left carotid artery (CILCA), and the different landing zones in each arch type. Results The CNN segmentation yielded a mean Dice score of 0.94 with respect to manual ground truth segmentations. Standard arches were characterized by significantly larger radius of curvature (p = 0.002) and lower tortuosity in zone 3 (p = 0.004) vs. CILCA arches. For both standard and CILCA arches, comparisons among PLZs revealed statistically significant differences in maximum zone diameters (p &amp;lt; 0.0001), angulation (p &amp;lt; 0.0001) and tortuosity (p &amp;lt; 0.0001). Conclusions We developed a CNN-based automated pipeline for the automated, and reliable geometric quantification of standard and CILCA aortic arches. This tool has the potential to support TEVAR pre-procedural planning in a real clinical setting. Abstract Figure. Automatic pipeline scheme

A transfer learning approach for automatic segmentation of the surgically treated anterior cruciate ligament.

Quantitative magnetic resonance imaging enables quantitative assessment of the healing anterior cruciate ligament or graft post-surgery, but its use is constrained by the need for time consuming manual image segmentation. The goal of this study was to validate a deep learning model for automatic segmentation of repaired and reconstructed anterior cruciate ligaments. We hypothesized that (1) a deep learning model would segment repaired ligaments and grafts with comparable anatomical similarity to intact ligaments, and (2) automatically derived quantitative features (i.e., signal intensity and volume) would not be significantly different from those obtained by manual segmentation. Constructive Interference in Steady State sequences were acquired of ACL repairs (n = 238) and grafts (n = 120). A previously validated model for intact ACLs was retrained on both surgical groups using transfer learning. Anatomical performance was measured with Dice coefficient, sensitivity, and precision. Quantitative features were compared to ground truth manual segmentation. Automatic segmentation of both surgical groups resulted in decreased anatomical performance compared to intact ACL automatic segmentation (repairs/grafts: Dice coefficient = .80/.78, precision = .79/.78, sensitivity = .82/.80), but neither decrease was statistically significant (Kruskal-Wallis: Dice coefficient p = .02, precision p = .09, sensitivity p = .17; Dunn post-hoc test for Dice coefficient: repairs/grafts p = .054/.051). There were no significant differences in quantitative features between the ground truth and automatic segmentation of repairs/grafts (0.82/2.7% signal intensity difference, p = .57/.26; 1.7/2.7% volume difference, p = .68/.72). The anatomical similarity performance and statistical similarities of quantitative features supports the use of this automated segmentation model in quantitative magnetic resonance imaging pipelines, which will accelerate research and provide a step towards clinical applicability.