Lung Cancer Segmentation Using an Enhanced TransUNet++ Architecture

Rapid and accurate muscle segmentation is essential for the diagnosis and monitoring of many musculoskeletal diseases. As gold standard, manual annotation suffers from intensive labor and high inter-operator reproducibility errors. In this study, deep learning (DL) based automatic muscle segmentation from MR scans is investigated for post-menopausal women, who normally experience a decline in muscle volume. The performance of four Deep Learning (DL) models was evaluated: U-Net and UNet++ and two modified U-Net networks, which combined feature fusion and attention mechanisms (Feature-Fusion-UNet, FFU, and Attention-Feature-Fusion-UNet, AFFU). The models were tested for automatic segmentation of 16-lower limb muscles from MRI scans of two cohorts of post-menopausal women (11 subjects in PMW-1, 8 subjects in PMW-2; from two different studies so considered independent datasets) and 10 obese post-menopausal women (PMW-OB). Furthermore, a novel data augmentation approach is proposed to enlarge the training dataset. The results were assessed and compared by using the Dice similarity coefficient (DSC), relative volume error (RVE), and Hausdorff distance (HD). The best performance among all four DL models was achieved by AFFU (PMW-1: DSC 0.828 ± 0.079, 1-RVE 0.859 ± 0.122, HD 29.9mm ± 26.5mm; PMW-2: DSC 0.833 ± 0.065, 1-RVE 0.873 ± 0.105, HD 25.9mm ± 27.9mm; PMW-OB: DSC 0.862 ± 0.048, 1-RVE 0.919 ± 0.076, HD 34.8mm ± 46.8mm). Furthermore, the augmentation of data significantly improved the DSC scores of U-Net and AFFU for all 16 tested muscles (between 0.23% and 2.17% (DSC), 1.6%-1.93% (1-RVE), and 9.6%-19.8% (HD) improvement). These findings highlight the feasibility of utilizing DL models for automatic segmentation of muscles in post-menopausal women and indicate that the proposed augmentation method can enhance the performance of models trained on small datasets.

To develop a novel automatic delineation model, the Multi-Scale Channel Attention U-Net (MCAU-Net) model, for gallbladder segmentation on CT images of patients with liver cancer. We retrospectively collected the CT images from 120 patients with liver cancer, based on which ground truth was manually delineated by physicians. The images and ground truth constitute a dataset, which was proportionally divided into a training set (54%), a validation set (6%), and a test set (40%). Data augmentation was performed on the training set. Our proposed MCAU-Net model was employed for gallbladder segmentation and its performance was evaluated using Dice Similarity Coefficient (DSC), Jaccard Similarity Coefficient (JSC), Positive Predictive Value (PPV), Sensitivity (SE), Hausdorff Distance (HD), Relative Volume Difference (RVD), and Volumetric Overlap Error (VOE) metrics. On the test set, MCAU-Net achieved DSC, JSC, PPV, SE, HD, RVD, and VOE values of 0.85 ± 0.22, 0.79 ± 0.23, 0.92 ± 0.14, 0.84 ± 0.23, 2.75 ± 0.98, 0.18 ± 0.48, and 0.22 ± 0.42, respectively. Compared to the control models, U-Net, SEU-Net and TransUNet, the MCAU-Net improved DSC 0.06, 0.04 and 0.06, JSC by 0.09, 0.06 and 0.09, PPV by 0.08, 0.08 and 0.05, SE by 0.05,0.05 and 0.07, and reduced HD by 0.45, 0.28 and 0.41, RVD by 0.07, 0.03 and 0.07, VOE by 0.04, 0.02 and 0.08 respectively. Qualitative results revealed that MCAU-Net produced smoother and more accurate boundaries, closer to the expert delineation, with less over-segmentation and under-segmentation and improved robustness. The MCAU-Net model significantly improves gallbladder segmentation on CT images. It satisfies clinical requirements and enhances the efficiency of physicians, particularly in segmenting complex anatomical structures.

Deep Learning for Contour Quality Assurance on RTOG 0933

Magnetic resonance (MR) imaging is the gold standard in image-guided brachytherapy (IGBT) due to its superior soft-tissue contrast for target and organs-at-risk (OARs) delineation. Accurate and fast segmentation of MR images are very important for high-quality IGBT treatment planning. The purpose of this work is to implement and evaluate deep learning (DL) models for the automatic segmentation of targets and OARs in MR image-based high-dose-rate (HDR) brachytherapy for cervical cancer. A 2D DL model using residual neural network architecture (ResNet50) was developed to contour the targets (gross tumor volume (GTV), high-risk clinical target volume (HR CTV), and intermediate-risk clinical target volume (IR CTV)) and OARs (bladder, rectum, sigmoid, and small intestine) automatically on axial MR slices of HDR brachytherapy patients. Furthermore, two additional 2D DL models using sagittal and coronal images were also developed. A 2.5D model was generated by combining the outputs from axial, sagittal, and coronal DL models. Similarly, a 2D and 2.5D DL models were also generated for the inception residual neural network (InceptionResNetv2 (InRN)) architecture. The geometric (Dice similarity coefficient (DSCs) and 95th percentile of Hausdorff distance (HD)) and dosimetric accuracy of 2D (axial only) and 2.5D (axial+sagittal+coronal) DL model generated contours were calculated and compared. The mean (range) DSCs of ResNet50 across all contours were 0.674 (0.05-0.96) and 0.715 (0.26-0.96) for the 2D and 2.5D models, respectively. For InRN, these were 0.676 (0.11-0.96) and 0.723 (0.35-0.97) for the 2D and 2.5D models, respectively. The mean HD of ResNet50 across all contours was 15.6mm (1.8-69mm) and 12.1mm (1.7-44mm) for the 2D and 2.5D models, respectively. The similar results for InRN were 15.4mm (2-68mm) and 10.3mm (2.7-39mm) for the 2D and 2.5D models, respectively. The dosimetric parameters (D90) of GTV and HR CTV for manually contoured plans matched better with the 2.5D model (p>0.6) and the results from the 2D model were slightly lower (p<0.08). On the other hand, the IR CTV doses (D90) for all of the models were slightly lower (2D: -1.3 to -1.5Gy and 2.5D: -0.5 to -0.6Gy) and the differences were statistically significant for the 2D model (2D: p<0.000002 and 2.5D: p>0.06). In case of OARs, the 2.5D model segmentations resulted in closer dosimetry than 2D models (2D: p=0.07-0.91 and 2.5D: p=0.16-1.0). The 2.5D DL models outperformed their respective 2D models for the automatic contouring of targets and OARs in MR image-based HDR brachytherapy for cervical cancer. The InceptionResNetv2 model performed slightly better than ResNet50.

Improving GI Toxicity Models Through Deep Learning-Based Segmentation and Biomechanical Model-Based Dose Accumulation

Introduction: Magnetic resonance imaging (MRI) is the most common tool to examine glioblastoma. Preoperative MRI can be used to initial diagnosis, and surgery planning, while follow-up MRIs can be used to evaluate treatment responses, identify recurrency, and detect side effects. The follow-up MRIs are usually taken after the first-line therapy, such as maximal safe resection. In the past, radiologists manually segment the tumor regions from normal brain tissue on follow-up MRIs, which is time-consuming, error-prone, and challenging. Several deep-learning (DL) models have been developed utilizing preoperative images, but their performance has yet to be evaluated on follow-up MRIs. In this research, we built the largest follow-up MRI cohort (311 patients) to assess these DL models and their generalizability and performance on independent preoperative and follow-up images. We also made the first follow-up-based deep learning models for this specific task. Methods: All evaluation deep learning models (10 models) were trained by the Brain Tumor Segmentation challenge 2020 (BraTS’20) and evaluated by fifty pairs of preoperative and follow-up scans from our institution. The segmentation form our institution is evaluated by board certified radiologist. MRIs in the BraTS’20 dataset were all preoperative scans. After the evaluation, we randomly assigned 264 patients’ scans from our institution to the training dataset and 47 patients’ scans to the testing dataset. We compared three types of models in our follow-up deep learning model, including 1) UNet-3D, 2) UNet-3D+transfer-learning, and 3) UNet-3D+transfer-learning+baysian-learning. The benchmark for all models was the Dice similarity coefficient (DSC). DSC can measure the spatial overlap between model prediction and ground truth. The value of DSC is between zero to one. Zero means no overlap and one indicates complete overlap. Results: Our study demonstrates that the BraTS'20 trained models' performance decreased by 13.05% in independent preoperative MRI scans and 19.04% in follow-up MRI scans. The most significant mismatch regions were FLAIR hyperintense regions (3.68% drop in independent preoperative scans and 10.61% drop in independent follow-up scans) and Non-enhancing core (5.20% drop in independent preoperative scans and 11.99% drop in independent follow-up scans). Our best model can achieve the best DSC among three tumor regions compared to all evaluation models (FLAIR hyperintense regions: DSC 0.77 V.S. DSC 0.58; Enhancing tumor region: DSC 0.87 V.S. DSC 0.68; Non-enhancing tumor region: DSC 0.92 V.S. DSC 0.55). Conclusion: Maximal safe resection induced brain structure change, decreasing the performance of the preoperative-based DL model. Implementing a follow-up MRI-based segmentation model is essential to make accurate and generalizable results to address structural changes after maximal safe resection. Our follow-up DL model demonstrates the DSC score can be recovered. We commit to further developing the tool to assist radiologists in handling follow-up MRIs for glioblastoma patients. Citation Format: Kang Lin Hsieh, Tanjida Kabir, Luis Nunez-Rubiano, Yu-Chun Hsu, Yu Cai, Juan Rodriguez Quintero, Octavio Arevalo, Kangyi Zhao, Jackie Zhang, Jiguang Zhu Zhu, Roy Riascos, Xioaqian Jiang, Shayan Shams. A confident and operator-independent deep segmentation model to measure residual tumor volume in the follow-up MRIs for glioblastoma [abstract]. In: Proceedings of the American Association for Cancer Research Annual Meeting 2023; Part 2 (Clinical Trials and Late-Breaking Research); 2023 Apr 14-19; Orlando, FL. Philadelphia (PA): AACR; Cancer Res 2023;83(8_Suppl):Abstract nr LB067.

End-to-end deep learning model with multi-channel and attention mechanisms for multi-class diagnosis in CT-T staging of advanced gastric cancer.

Interpreting medical images is certainly a complex task which requires extensive knowledge [1]. According to Computer Aided Diagnosis (CAD) serves as a second opinion that will help radiologists in diagnosis and on the other hand Content-based Image Retrieval uses visual content to help users browse, search and retrieve similar medical images from a database based on the user’s interest [2-4]. The competency of the CBMIR system depends on feature extraction methods [5]. The textural features are very important to determine the content of a medical image. Textural features provide scenic depth, the spatial distribution of tonal variation, and surface orientation [6]. Therefore, this study seeks to compare and evaluate some of the hand-crafted texture feature extraction techniques in CBMIR. This is to help those concerned in enhancing CBIR systems to make informed decisions concerning the selection of the best textural feature extraction techniques. Since there is no clear indication of which of the various texture feature extraction techniques is best suited for a given performance metric when considering which of the techniques to choose for a particular study in CBMIR systems. The objective of this work, therefore, is to comparatively evaluate the performance of the following texture feature extraction techniques; Local Binary Pattern (LBP), Gabor Filter, Gray-Level Co-occurrence Matrix (GLCM), Haralick Descriptor, Features from Accelerated Segment Test (FAST) and Features from Accelerated Segment Test and Binary Robust Independent Elementary Features (FAST &amp;BRIEF) using the metrics; precision, recall, f1-score, mean squared error (MSE), accuracy and time. These techniques are coupled with specific similarity measure to obtain results. The results showed that LBP, Haralick Descriptor, FAST, and GLCM had the best results in terms of (Precision and Accuracy), Time, F1-Score, and Recall respectively. LBP had 82.05% and 88.23% scores for precision and accuracy respectively. The following scores represent the performance of the Haralick descriptor, FAST, and GLCM models respectively; 0.88s, 38.7%, and 44.82%. These test scores are obtained from datasets ranging from 1k-10.5k. Aside from LBP outperforming the other 5 models mentioned, it still outperformed the following proposed models ' [7]’, ‘Tamura texture feature and wavelet transform combined with Hausdorff distance - [8]’, ‘ [9]’ in terms of (precision, accuracy, and recall) and (precision and recall) respectively and probably f1-score (since f1-score is the weighted average of precision and recall). It is believed that an ensemble of LBP, Haralick descriptors, and Support Vector Machine (SVM) can represent a robust system for both medical image retrieval and classification.

Simple SummaryWith advances in radiotherapy (RT) technique and more frequent use of stereotactic ablative radiotherapy (SABR), precise segmentation of all brain metastases (BM) including a small volume of BM is essential to choose an appropriate treatment modality. However, the process of detecting and manually delineating BM with small volumes often results in missing delineation and requires a great amount of labor. To address this issue, we present a useful deep learning (DL) model for the detection and segmentation of BMwith contrast-enhanced magnetic resonance images. Specifically, we applied effective training techniques to detect and segment a BM of less than 0.04 cc, which is relatively small compared to previous studies. The results of our DL model demonstrated that the proposed methods provide considerable benefit for BM, even small-volume BM, detection, and segmentation for SABR.Recently, several efforts have been made to develop the deep learning (DL) algorithms for automatic detection and segmentation of brain metastases (BM). In this study, we developed an advanced DL model to BM detection and segmentation, especially for small-volume BM. From the institutional cancer registry, contrast-enhanced magnetic resonance images of 65 patients and 603 BM were collected to train and evaluate our DL model. Of the 65 patients, 12 patients with 58 BM were assigned to test-set for performance evaluation. Ground-truth for BM was assigned to one radiation oncologist to manually delineate BM and another one to cross-check. Unlike other previous studies, our study dealt with relatively small BM, so the area occupied by the BM in the high-resolution images were small. Our study applied training techniques such as the overlapping patch technique and 2.5-dimensional (2.5D) training to the well-known U-Net architecture to learn better in smaller BM. As a DL architecture, 2D U-Net was utilized by 2.5D training. For better efficacy and accuracy of a two-dimensional U-Net, we applied effective preprocessing include 2.5D overlapping patch technique. The sensitivity and average false positive rate were measured as detection performance, and their values were 97% and 1.25 per patient, respectively. The dice coefficient with dilation and 95% Hausdorff distance were measured as segmentation performance, and their values were 75% and 2.057 mm, respectively. Our DL model can detect and segment BM with small volume with good performance. Our model provides considerable benefit for clinicians with automatic detection and segmentation of BM for stereotactic ablative radiotherapy.

BackgroundNasopharyngeal carcinoma (NPC), an endemic malignancy in Southeast Asia and southern China, necessitates precise delineation of the gross tumor volume (GTV) for radiotherapy to optimize patient prognosis. Positron emission tomography (PET) imaging offers valuable metabolic insights to guide radiotherapy planning but is hampered by manual GTV segmentation, which is labor-intensive and prone to inter- and intra-observer variability. Conventional segmentation methods are inherently limited, and comparative evaluations of deep learning (DL) frameworks for PET-based primary NPC lesion segmentation remain scarce. Thus, this study aimed to optimize the delineation of primary NPC radiotherapy target volumes by leveraging PET-image-based DL segmentation technology, with the goal of enhancing both the precision and efficiency of lesion delineation.MethodsResearchers retrospectively collected PET imaging data from 212 NPC patients at the Affiliated Hospital of Guangdong Medical University. The patients were randomly divided into training (170 patients) and testing (42 patients) datasets. A radiation oncologist and a nuclear medicine physician collaboratively delineated the cancer lesion boundaries through consensus. After data preprocessing, three DL models (Res-Unet, Nn-Unet, and Nn-Former) were used to automatically segment the lesions. Training was based on the training dataset. During evaluation, the models’ segmentation performance was comprehensively assessed using Dice similarity coefficient (DSC) and Hausdorff distance (HD). A visual analysis of the results was also conducted to intuitively understand the models’ segmentation capabilities.ResultsIn the training set, Nn-Unet achieved the highest DSC of 0.869, whereas in the testing set, its DSC was 0.833. The DSC for Nn-Former hovered around 0.8 in both the training and testing sets. In contrast, Res-Unet demonstrated the lowest DSC values among the three models, particularly in the testing set (DSC =0.794). Statistical analysis revealed that the DSC of Nn-Unet was significantly higher than that of both Res-Unet and Nn-Former in the training set (P<0.01 for both comparisons). Regarding the 95% HD and average surface distance (ASD) values, Nn-Unet outperformed the other two models in both the training and testing sets. However, Res-Unet exhibited a much higher HD value in the testing set compared to the training set. The Loss curves for all models gradually decreased as training progressed, indicating that the models were learning relevant features of NPC lesions. The final Loss value for Res-Unet was approximately 0.7, whereas Nn-Unet and Nn-Former had final Loss values below 0.8.ConclusionsThe PET-image-based automatic segmentation model for the primary tumor of NPC established in this study demonstrates the clinical potential of Nn-Unet in primary lesion of NPC segmentation tasks.

Background:The choroid plexus functions as the blood-cerebrospinal fluid barrier, plays an important role in neurofluid production and circulation, and has gained increased attention in light of the recent elucidation of neurofluid circulation dysfunction in neurodegenerative conditions. However, methods for routinely quantifying choroid plexus volume are suboptimal and require technical improvements and validation. Here, we propose three deep learning models that can segment the choroid plexus from commonly-acquired anatomical MRI data and report performance metrics and changes across the adult lifespan.Methods:Fully convolutional neural networks were trained from 3-D T1-weighted, 3-D T2-weighted, and 2-D T2-weighted FLAIR MRI and gold-standard manual segmentations in healthy and neurodegenerative participants across the lifespan (n=50; age=21–85 years). Dice coefficients, 95% Hausdorff distances, and area-under-curve (AUCs) were calculated for each model and compared to segmentations from FreeSurfer using two-tailed Wilcoxon tests (significance criteria: p<0.05 after false discovery rate multiple comparisons correction). Metrics were regressed against lateral ventricular volume using generalized linear models to assess model performance for varying levels of atrophy. Finally, models were applied to an expanded cohort of healthy adults (n=98; age=21–89 years) to provide an exemplar of choroid plexus volumetry values across the lifespan.Results:Deep learning results yielded Dice coefficient=0.72, Hausdorff distance=1.97 mm, AUC=0.87 for T1-weighted MRI, Dice coefficient=0.72, Hausdorff distance=2.22 mm, AUC=0.87 for T2-weighted MRI, and Dice coefficient=0.74, Hausdorff distance=1.69 mm, AUC=0.87 for T2-weighted FLAIR MRI; values did not differ significantly between2 MRI sequences and were statistically improved compared to current commercially-available algorithms (p<0.001). The intraclass coefficients were 0.95, 0.95, and 0.96 between T1-weighted and T2-FLAIR, T1-weighted and T2-weighted, and T2-weighted and T2-FLAIR models, respectively. Mean lateral ventricle choroid plexus volume across all participants was 3.20±1.4 cm3; a significant, positive relationship (R2=0.54; slope=0.047) was observed between participant age and choroid plexus volume for all MRI sequences (p<0.001).Conclusions:Findings support comparable performance in choroid plexus delineation between standard, clinically available, non-contrasted anatomical MRI sequences. The software embedding the evaluated models is freely available online and should provide a useful tool for the growing number of studies that desire to quantitatively evaluate choroid plexus structure and function (https://github.com/hettk/chp_seg).

BackgroundThe choroid plexus functions as the blood-cerebrospinal fluid (CSF) barrier, plays an important role in CSF production and circulation, and has gained increased attention in light of the recent elucidation of CSF circulation dysfunction in neurodegenerative conditions. However, methods for routinely quantifying choroid plexus volume are suboptimal and require technical improvements and validation. Here, we propose three deep learning models that can segment the choroid plexus from commonly-acquired anatomical MRI data and report performance metrics and changes across the adult lifespan.MethodsFully convolutional neural networks were trained from 3D T1-weighted, 3D T2-weighted, and 2D T2-weighted FLAIR MRI using gold-standard manual segmentations in control and neurodegenerative participants across the lifespan (n = 50; age = 21–85 years). Dice coefficients, 95% Hausdorff distances, and area-under-curve (AUCs) were calculated for each model and compared to segmentations from FreeSurfer using two-tailed Wilcoxon tests (significance criteria: p < 0.05 after false discovery rate multiple comparisons correction). Metrics were regressed against lateral ventricular volume using generalized linear models to assess model performance for varying levels of atrophy. Finally, models were applied to an expanded cohort of adult controls (n = 98; age = 21–89 years) to provide an exemplar of choroid plexus volumetry values across the lifespan.ResultsDeep learning results yielded Dice coefficient = 0.72, Hausdorff distance = 1.97 mm, AUC = 0.87 for T1-weighted MRI, Dice coefficient = 0.72, Hausdorff distance = 2.22 mm, AUC = 0.87 for T2-weighted MRI, and Dice coefficient = 0.74, Hausdorff distance = 1.69 mm, AUC = 0.87 for T2-weighted FLAIR MRI; values did not differ significantly between MRI sequences and were statistically improved compared to current commercially-available algorithms (p < 0.001). The intraclass coefficients were 0.95, 0.95, and 0.96 between T1-weighted and T2-weighted FLAIR, T1-weighted and T2-weighted, and T2-weighted and T2-weighted FLAIR models, respectively. Mean lateral ventricle choroid plexus volume across all participants was 3.20 ± 1.4 cm3; a significant, positive relationship (R2 = 0.54-0.60) was observed between participant age and choroid plexus volume for all MRI sequences (p < 0.001).ConclusionsFindings support comparable performance in choroid plexus delineation between standard, clinically available, non-contrasted anatomical MRI sequences. The software embedding the evaluated models is freely available online and should provide a useful tool for the growing number of studies that desire to quantitatively evaluate choroid plexus structure and function (https://github.com/hettk/chp_seg).

The vessel-plaque interface detection and lesion length of coronary artery disease are still challenging using the deep learning with intravascular ultrasound (IVUS). We used AI-based image segmentation technology to develop a deep learning model for classifying frames with IVUS-derived target lesion. Methods. A total of 3,782 target lesion frames from 28 acute coronary syndrome(ACS) patients were randomized into training and test sets in a 2:1 ratio. The training algorithm was based on the measurement of the core lab. A segmentation model using non-local blocks was used to increase the similarity between pixels of the same class in a U-Net-based model. The edge detection was performed on the region of interest, and the detected edge was used as an aid in deriving the final segmentation result. And the morphological guided measurements were used to provide the information about the vessel area, lumen area, plaque burden and lesion length. Quantitative evaluation was conducted on four scales: Jaccard Measure (JM), Dice Similarity Coefficient (DSC), Hausdorff Distance (HD), and Percentage of Area Difference (PAD). And we compared the diagnostic accuracy to detect the vessel-plaque interface of the target lesion. Results. In the training sample (1,256 IVUS frames of target lesion), and the test samples (1,072 IVUS frames of target lesion) were used for the proposed model. The external elastic membrane area (EEM), lumen area, plaque area and thrombus were assessed with the proposed algorithm. The measurement of EEM was DSC 0.99, JM 0.98, HD 0.06, and PAD 0.01. And the lumen also shows excellent performance with DSC 0.98, JM 0.97, HD 0.08, and PAD 0.01. In addition, the proposed model was compared with ground-truth for Thrombus, and showed high values of 0.88 for DSC, 0.83 for JM, 0.21 for HD, and 0.09 for PAD. Conclusion. An advanced algorithm can accurately detect the vessel-plaque interface and estimated stent length of coronary artery disease with high reproducibility. F

MP09-18 DEEP LEARNING ALGORITHMS FOR MULTI-REGION BLADDER CANCER SEGMENTATION

Deep learning model for automated segmentation of sphenoid sinus and middle skull base structures in CBCT volumes using nnU-Net v2.