Brain Tumor Segmentation Challenge Research Articles

Repurposing the Public BraTS Dataset for Postoperative Brain Tumour Treatment Response Monitoring.

The Brain Tumor Segmentation (BraTS) Challenge has been a main driver of the development of deep learning (DL) algorithms and provides by far the largest publicly available expert-annotated brain tumour dataset but contains solely preoperative examinations. The aim of our study was to facilitate the use of the BraTS dataset for training DL brain tumour segmentation algorithms for a postoperative setting. To this end, we introduced an automatic conversion of the three-label BraTS annotation protocol to a two-label annotation protocol suitable for postoperative brain tumour segmentation. To assess the viability of the label conversion, we trained a DL algorithm using both the three-label and the two-label annotation protocols. We assessed the models pre- and postoperatively and compared the performance with a state-of-the-art DL method. The DL algorithm trained using the BraTS three-label annotation misclassified parts of 10 out of 41 fluid-filled resection cavities in 72 postoperative glioblastoma MRIs, whereas the two-label model showed no such inaccuracies. The tumour segmentation performance of the two-label model both pre- and postoperatively was comparable to that of a state-of-the-art algorithm for tumour volumes larger than 1 cm3. Our study enables using the BraTS dataset as a basis for the training of DL algorithms for postoperative tumour segmentation.

LightNet: A Novel Lightweight Convolutional Network for Brain Tumor Segmentation in Healthcare.

Diagnosis, treatment planning, surveillance, and the monitoring of clinical trials for brain diseases all benefit greatly from neuroimaging-based tumor segmentation. Recently, Convolutional Neural Networks (CNNs) have demonstrated promising results in enhancing the efficiency of image-based brain tumor segmentation. Most current work on CNNs, however, is devoted to creating increasingly complicated convolution modules to improve performance, which in turn raises the computing cost of the model. This work proposes a simple and effective feed-forward CNN, LightNet (Light Network). Based on multi-path and multi-level, it replaces traditional convolutional methods with light operations, which reduces network parameters and redundant feature maps. In the up-sampling stage, a light channel attention module is added to achieve richer multi-scale and spatial semantic feature information extraction of brain tumor. The performance of the network is evaluated in the Multimodal Brain Tumor Segmentation Challenge (BraTS 2015) dataset, and results are presented here alongside other high-performing CNNs. Results show comparable accuracy with other methods but with increased efficiency, segmentation performance, and reduced redundancy and computational complexity. The result is a high-performing network with a balance between efficiency and accuracy, allowing, for example, better energy performance on mobile devices.

SwinVNETR: Swin V-net Transformer with non-local block for volumetric MRI Brain Tumor Segmentation

Brain Tumor Segmentation (BTS) and classification are important and growing research fields. Magnetic resonance imaging (MRI) is commonly used in the diagnosis of brain tumours owing to its low radiation exposure and high image quality. One of the current subjects in the field of medical imaging is how to quickly and precisely segment MRI scans of brain tumours. Unfortunately, most existing brain tumour segmentation algorithms use inadequate 2D picture segmentation methods and fail to capture the spatial correlation between features. In this study, we propose a segmentation model (SwinVNETR) Swin V-NetTRansformer-based architecture with a non-local block. This model was trained using the Brain Tumor Segmentation Challenge BraTS 2021 dataset. The Dice similarity coefficients for the enhanced tumour (ET), whole tumour (WT), and tumour core (TC) are 0.84, 0.91, and 0.87, respectively. By leveraging this methodology, we can segment brain tumours more accurately than ever before. In conclusion, we present the findings of our model through the application of the Grad-CAM methodology, an eXplainable Artificial Intelligence (XAI) technique utilized to elucidate the insights derived from the model, which helped in better understanding; doctors can better diagnose and treat patients with brain tumours.

Optimizing anomaly detection in 3D MRI scans: The role of ConvLSTM in medical image analysis

The analysis of Medical Images (MI), particularly the detection and classification of anomalies in 3D MRI (Magnetic Resonance Imaging) scans, plays a critical part in timely intervention and personalized therapy plans. In our paper, a comprehensive methodology for anomaly detection in 3D MRI scans of the brain is proposed that combines advanced Deep Learning (DL) techniques, including Convolutional Long Short-Term Memory (ConvLSTM) model, with efficient statistical feature extraction from segmented anomaly regions. The data collection phase utilizes three main datasets namely the Brain Tumor Segmentation Challenge (BRATS), the Federated Tumor Segmentation Challenge (FETS), and the Medical Segmentation Decathlon (MSD). The research begins with preprocessing steps including image resizing, intensity normalization, and alignment of anomaly region segmentation masks. The targeted anomaly regions within the samples are segmented using U-Net architecture and then follow statistical feature extraction procedure. Dimensionality reduction methods such as Principal Component Analysis (PCA) and Recursive Feature Elimination (RFE) are utilized to streamline the feature space. The ConvLSTM is then used to classify the anomalies using both convolution and recurrent layers to capture spatiotemporal patterns in MRI data. The model is fine-tuned and iterated for better classification performance using Adam optimizer. The statistical evaluation results with accuracy of 98.9 % showed that the designed ConvLSTM method is more suitable for clinical diagnosis and treatment design in detecting anomalies in 3D MRI images.

IMG-28. AUTOMATIC BRAIN TUMOR VOLUMETRIC ANALYSIS IN MAGNETIC RESONANCE IMAGING GENERALIZABLE TO PEDIATRIC NEURO-ONCOLOGY

Abstract BACKGROUND The prognosis of brain tumors is variable in clinical practice if it only relies on human interpretation of magnetic resonance imaging (MRI). The automatic segmentation of brain tumors in MRI enables quantitative analysis in support of clinical trials and personalized patient care. We developed benchmarked deep learning-based tools that are generalizable to the volumetric quantification of various tumor types across diverse populations. METHODS We participated in the well-established international brain tumor segmentation challenge (BraTS 2023) and benchmarking competition. The challenge made available 4,500 multi-national brain tumor cases with multi-sequence MRIs, including pediatric high-grade gliomas (PED), i.e., high-grade astrocytoma and diffuse midline glioma, and adult gliomas, brain metastases (MET) and intracranial meningiomas (MEN). Each case comprises four MRI volumes: T1, contrast-enhanced T1, T2, and T2-FLAIR. Manual segmentations were provided to establish ground truth for enhancing tumor (ET), tumor core (TC), and whole tumor (WT). Our framework used a model ensemble strategy based on two state-of-the-art deep learning models: a convolutional neural network (nnU-Net) and a vision transformer (Swin UNETR) and was tested for broader applicability across multiple tumor types. The framework was trained on 99, 1,000, and 165 cases and validated on 45, 141, and 31 unseen cases for PED, MEN, and MET, respectively. Automatic segmentations were evaluated by lesion-wise volume overlap (Dice similarity score, DSC) and Hausdorff distance (HD). RESULTS In the evaluation on independent unseen test datasets, our automatic tool was ranked first for PED, third for MEN, and fourth for MET volumetric analysis. Our method resulted in PED lesion-wise DSC of 0.733, 0.782, 0.817 and HD (mm) of 75.93, 25.54, 24.18 for ET, TC, and WT, respectively. CONCLUSIONS These brain tumor volumetric analysis tools are readily available to be efficiently tested on diverse datasets. Automatic MRI analysis provides consistent quantitative data for multi-institutional protocols and clinical trials.

Deep learning automatic semantic segmentation of glioblastoma multiforme regions on multimodal magnetic resonance images.

In patients having naïve glioblastoma multiforme (GBM), this study aims to assess the efficacy of Deep Learning algorithms in automating the segmentation of brain magnetic resonance (MR) images to accurately determine 3D masks for 4 distinct regions: enhanced tumor, peritumoral edema, non-enhanced/necrotic tumor, and total tumor. A 3D U-Net neural network algorithm was developed for semantic segmentation of GBM. The training dataset was manually delineated by a group of expert neuroradiologists on MR images from the Brain Tumor Segmentation Challenge 2021 (BraTS2021) image repository, as ground truth labels for diverse glioma (GBM and low-grade glioma) subregions across four MR sequences (T1w, T1w-contrast enhanced, T2w, and FLAIR) in 1251 patients. The in-house test was performed on 50GBM patients from our cohort (PerProGlio project). By exploring various hyperparameters, the network's performance was optimized, and the most optimal parameter configuration was identified. The assessment of the optimized network's performance utilized Dice scores, precision, and sensitivity metrics. Our adaptation of the 3D U-net with additional residual blocks demonstrated reliable performance on both the BraTS2021 dataset and the in-house PerProGlio cohort, employing only T1w-ce sequences for enhancement and non-enhanced/necrotic tumor models and T1w-ce + T2w + FLAIR for peritumoral edema and total tumor. The mean Dice scores (training and test) were 0.89 and 0.75; 0.75 and 0.64; 0.79 and 0.71; and 0.60 and 0.55, for total tumor, edema, enhanced tumor, and non-enhanced/necrotic tumor, respectively. The results underscore the high precision with which our network can effectively segment GBM tumors and their distinct subregions. The level of accuracy achieved agrees with the coefficients recorded in previous GBM studies. In particular, our approach allows model specialization for each of the different tumor subregions employing only those MR sequences that provide value for segmentation.

TACA-RNet: Tri-Axis Based Context-Aware Reverse Network for Multimodal Brain Tumor Segmentation

Brain tumor segmentation using Magnetic Resonance Imaging (MRI) is vital for clinical decision making. Traditional deep learning-based studies using convolutional neural networks have predominantly processed MRI data as two-dimensional slices, leading to the loss of contextual information. While three-dimensional (3D) convolutional layers represent an advancement, they have not fully exploited pathological information according to the three-axis nature of 3D MRI data—axial, coronal, and sagittal. Recognizing these limitations, we introduce a Tri-Axis based Context-Aware Reverse Network (TACA-RNet). This innovative approach leverages the unique 3D spatial orientations of MRI, learning crucial information on brain anatomy and pathology. We incorporated three specialized modules: a Tri-Axis Channel Reduction module for optimizing feature dimensions, a MultiScale Contextual Fusion module for aggregating multi-scale features and enhancing spatial discernment, and a 3D Axis Reverse Attention module for the precise delineation of tumor boundaries. The TACA-RNet leverages three specialized modules to enhance the understanding of tumor characteristics and spatial relationships within MRI data by fully utilizing its tri-axial structure. Validated on the Brain Tumor Segmentation Challenge 2018 and 2020 datasets, the TACA-RNet demonstrated superior performances over contemporary methodologies. This underscores the critical role of leveraging the three-axis structure of MRI to enhance segmentation accuracy.

Training and Comparison of nnU-Net and DeepMedic Methods for Autosegmentation of Pediatric Brain Tumors.

Tumor segmentation is essential in surgical and treatment planning and response assessment and monitoring in pediatric brain tumors, the leading cause of cancer-related death among children. However, manual segmentation is time-consuming and has high interoperator variability, underscoring the need for more efficient methods. After training, we compared 2 deep-learning-based 3D segmentation models, DeepMedic and nnU-Net, with pediatric-specific multi-institutional brain tumor data based on multiparametric MR images. Multiparametric preoperative MR imaging scans of 339 pediatric patients (n = 293 internal and n = 46 external cohorts) with a variety of tumor subtypes were preprocessed and manually segmented into 4 tumor subregions, ie, enhancing tumor, nonenhancing tumor, cystic components, and peritumoral edema. After training, performances of the 2 models on internal and external test sets were evaluated with reference to ground truth manual segmentations. Additionally, concordance was assessed by comparing the volume of the subregions as a percentage of the whole tumor between model predictions and ground truth segmentations using the Pearson or Spearman correlation coefficients and the Bland-Altman method. The mean Dice score for nnU-Net internal test set was 0.9 (SD, 0.07) (median, 0.94) for whole tumor; 0.77 (SD, 0.29) for enhancing tumor; 0.66 (SD, 0.32) for nonenhancing tumor; 0.71 (SD, 0.33) for cystic components, and 0.71 (SD, 0.40) for peritumoral edema, respectively. For DeepMedic, the mean Dice scores were 0.82 (SD, 0.16) for whole tumor; 0.66 (SD, 0.32) for enhancing tumor; 0.48 (SD, 0.27) for nonenhancing tumor; 0.48 (SD, 0.36) for cystic components, and 0.19 (SD, 0.33) for peritumoral edema, respectively. Dice scores were significantly higher for nnU-Net (P ≤ .01). Correlation coefficients for tumor subregion percentage volumes were higher (0.98 versus 0.91 for enhancing tumor, 0.97 versus 0.75 for nonenhancing tumor, 0.98 versus 0.80 for cystic components, 0.95 versus 0.33 for peritumoral edema in the internal test set). Bland-Altman plots were better for nnU-Net compared with DeepMedic. External validation of the trained nnU-Net model on the multi-institutional Brain Tumor Segmentation Challenge in Pediatrics (BraTS-PEDs) 2023 data set revealed high generalization capability in the segmentation of whole tumor, tumor core (a combination of enhancing tumor, nonenhancing tumor, and cystic components), and enhancing tumor with mean Dice scores of 0.87 (SD, 0.13) (median, 0.91), 0.83 (SD, 0.18) (median, 0.89), and 0.48 (SD, 0.38) (median, 0.58), respectively. The pediatric-specific data-trained nnU-Net model is superior to DeepMedic for whole tumor and subregion segmentation of pediatric brain tumors.

End-to-End Multi-task Learning Architecture for Brain Tumor Analysis with Uncertainty Estimation in MRI Images.

Brain tumors are a threat to life for every other human being, be it adults or children. Gliomas are one of the deadliest brain tumors with an extremely difficult diagnosis. The reason is their complex and heterogenous structure which gives rise to subjective as well as objective errors. Their manual segmentation is a laborious task due to their complex structure and irregular appearance. To cater to all these issues, a lot of research has been done and is going on to develop AI-based solutions that can help doctors and radiologists in the effective diagnosis of gliomas with the least subjective and objective errors, but an end-to-end system is still missing. An all-in-one framework has been proposed in this research. The developed end-to-end multi-task learning (MTL) architecture with a feature attention module can classify, segment, and predict the overall survival of gliomas by leveraging task relationships between similar tasks. Uncertainty estimation has also been incorporated into the framework to enhance the confidence level of healthcare practitioners. Extensive experimentation was performed by using combinations of MRI sequences. Brain tumor segmentation (BraTS) challenge datasets of 2019 and 2020 were used for experimental purposes. Results of the best modelwith four sequences show 95.1% accuracy for classification, 86.3% dicescore for segmentation, and a mean absolute error (MAE) of 456.59 for survival predictionon the test data. It is evident from the results that deep learning-based MTL models have the potential to automate the whole brain tumor analysis process and give efficient results with least inference time without human intervention. Uncertainty quantification confirms the idea that more data can improve the generalization ability and in turn can produce more accurate results with less uncertainty. The proposed model has the potential to be utilized in a clinical setup for the initial screening of glioma patients.

Fully Automatic Glioma Segmentation Algorithm of Magnetic Resonance Imaging Based on 3D-UNet With More Global Contextual Feature Extraction: An Improvement on Insufficient Extraction of Global Features

The fully automatic segmentation of glioma and its subregions is fundamental for computer-aided clinical diagnosis of tumors. In the segmentation process of brain magnetic resonance imaging (MRI), convolutional neural networks with small convolutional kernels can only capture local features and are ineffective at integrating global features, which narrows the receptive field and leads to insufficient segmentation accuracy. This study aims to use dilated convolution to address the problem of inadequate global feature extraction in 3D-UNet. 1) Algorithm construction: A 3D-UNet model with three pathways for more global contextual feature extraction, or 3DGE-UNet, was proposed in the paper. By using publicly available datasets from the Brain Tumor Segmentation Challenge (BraTS) of 2019 (335 patient cases), a global contextual feature extraction (GE) module was designed. This module was integrated at the first, second, and third skip connections of the 3D UNet network. The module was utilized to fully extract global features at different scales from the images. The global features thus extracted were then overlaid with the upsampled feature maps to expand the model's receptive field and achieve deep fusion of features at different scales, thereby facilitating end-to-end automatic segmentation of brain tumors. 2) Algorithm validation: The image data were sourced from the BraTs 2019 dataset, which included the preoperative MRI images of 335 patients across four modalities (T1, T1ce, T2, and FLAIR) and a tumor image with annotations made by physicians. The dataset was divided into the training, the validation, and the testing sets at an 8∶1∶1 ratio. Physician-labelled tumor images were used as the gold standard. Then, the algorithm's segmentation performance on the whole tumor (WT), tumor core (TC), and enhancing tumor (ET) was evaluated in the test set using the Dice coefficient (for overall effectiveness evaluation), sensitivity (detection rate of lesion areas), and 95% Hausdorff distance (segmentation accuracy of tumor boundaries). The performance was tested using both the 3D-UNet model without the GE module and the 3DGE-UNet model with the GE module to internally validate the effectiveness of the GE module setup. Additionally, the performance indicators were evaluated using the 3DGE-UNet model, ResUNet, UNet++, nnUNet, and UNETR, and the convergence of these five algorithm models was compared to externally validate the effectiveness of the 3DGE-UNet model. 1) In internal validation, the enhanced 3DGE-UNet model achieved Dice mean values of 91.47%, 87.14%, and 83.35% for segmenting the WT, TC, and ET regions in the test set, respectively, producing the optimal values for comprehensive evaluation. These scores were superior to the corresponding scores of the traditional 3D-UNet model, which were 89.79%, 85.13%, and 80.90%, indicating a significant improvement in segmentation accuracy across all three regions (P<0.05). Compared with the 3D-UNet model, the 3DGE-UNet model demonstrated higher sensitivity for ET (86.46% vs. 80.77%) (P<0.05) , demonstrating better performance in the detection of all the lesion areas. When dealing with lesion areas, the 3DGE-UNet model tended to correctly identify and capture the positive areas in a more comprehensive way, thereby effectively reducing the likelihood of missed diagnoses. The 3DGE-UNet model also exhibited exceptional performance in segmenting the edges of WT, producing a mean 95% Hausdorff distance superior to that of the 3D-UNet model (8.17 mm vs. 13.61 mm, P<0.05). However, its performance for TC (8.73 mm vs. 7.47 mm) and ET (6.21 mm vs. 5.45 mm) was similar to that of the 3D-UNet model. 2) In the external validation, the other four algorithms outperformed the 3DGE-UNet model only in the mean Dice for TC (87.25%), the mean sensitivity for WT (94.59%), the mean sensitivity for TC (86.98%), and the mean 95% Hausdorff distance for ET (5.37 mm). Nonetheless, these differences were not statistically significant (P>0.05). The 3DGE-UNet model demonstrated rapid convergence during the training phase, outpacing the other external models. The 3DGE-UNet model can effectively extract and fuse feature information on different scales, improving the accuracy of brain tumor segmentation.

Synthesizing Contrast-Enhanced MR Images from Noncontrast MR Images Using Deep Learning.

Recent developments in deep learning methods offer a potential solution to the need for alternative imaging methods due to concerns about the toxicity of gadolinium-based contrast agents. The purpose of the study was to synthesize virtual gadolinium contrast-enhanced T1-weighted MR images from noncontrast multiparametric MR images in patients with primary brain tumors by using deep learning. We trained and validated a deep learning network by using MR images from 335 subjects in the Brain Tumor Segmentation Challenge 2019 training data set. A held out set of 125 subjects from the Brain Tumor Segmentation Challenge 2019 validation data set was used to test the generalization of the model. A residual inception DenseNet network, called T1c-ET, was developed and trained to simultaneously synthesize virtual contrast-enhanced T1-weighted (vT1c) images and segment the enhancing portions of the tumor. Three expert neuroradiologists independently scored the synthesized vT1c images by using a 3-point Likert scale, evaluating image quality and contrast enhancement against ground truth T1c images (1 = poor, 2 = good, 3 = excellent). The synthesized vT1c images achieved structural similarity index, peak signal-to-noise ratio, and normalized mean square error scores of 0.91, 64.35, and 0.03, respectively. There was moderate interobserver agreement between the 3 raters, regarding the algorithm's performance in predicting contrast enhancement, with a Fleiss kappa value of 0.61. Our model was able to accurately predict contrast enhancement in 88.8% of the cases (scores of 2 to 3 on the 3-point scale). We developed a novel deep learning architecture to synthesize virtual postcontrast enhancement by using only conventional noncontrast brain MR images. Our results demonstrate the potential of deep learning methods to reduce the need for gadolinium contrast in the evaluation of primary brain tumors.

High-resolution MRI synthesis using a data-driven framework with denoising diffusion probabilistic modeling

Objective. High-resolution magnetic resonance imaging (MRI) can enhance lesion diagnosis, prognosis, and delineation. However, gradient power and hardware limitations prohibit recording thin slices or sub-1 mm resolution. Furthermore, long scan time is not clinically acceptable. Conventional high-resolution images generated using statistical or analytical methods include the limitation of capturing complex, high-dimensional image data with intricate patterns and structures. This study aims to harness cutting-edge diffusion probabilistic deep learning techniques to create a framework for generating high-resolution MRI from low-resolution counterparts, improving the uncertainty of denoising diffusion probabilistic models (DDPM). Approach. DDPM includes two processes. The forward process employs a Markov chain to systematically introduce Gaussian noise to low-resolution MRI images. In the reverse process, a U-Net model is trained to denoise the forward process images and produce high-resolution images conditioned on the features of their low-resolution counterparts. The proposed framework was demonstrated using T2-weighted MRI images from institutional prostate patients and brain patients collected in the Brain Tumor Segmentation Challenge 2020 (BraTS2020). Main results. For the prostate dataset, the bicubic interpolation model (Bicubic), conditional generative-adversarial network (CGAN), and our proposed DDPM framework improved the noise quality measure from low-resolution images by 4.4%, 5.7%, and 12.8%, respectively. Our method enhanced the signal-to-noise ratios by 11.7%, surpassing Bicubic (9.8%) and CGAN (8.1%). In the BraTS2020 dataset, the proposed framework and Bicubic enhanced peak signal-to-noise ratio from resolution-degraded images by 9.1% and 5.8%. The multi-scale structural similarity indexes were 0.970 ± 0.019, 0.968 ± 0.022, and 0.967 ± 0.023 for the proposed method, CGAN, and Bicubic, respectively. Significance. This study explores a deep learning-based diffusion probabilistic framework for improving MR image resolution. Such a framework can be used to improve clinical workflow by obtaining high-resolution images without penalty of the long scan time. Future investigation will likely focus on prospectively testing the efficacy of this framework with different clinical indications.

Analysis of Hybrid Feature Optimization Techniques Based on the Classification Accuracy of Brain Tumor Regions Using Machine Learning and Further Evaluation Based on the Institute Test Data.

The goal of this study was to get optimal brain tumor features from magnetic resonance imaging (MRI) images and classify them based on the three groups of the tumor region: Peritumoral edema, enhancing-core, and necrotic tumor core, using machine learning classification models. This study's dataset was obtained from the multimodal brain tumor segmentation challenge. A total of 599 brain MRI studies were employed, all in neuroimaging informatics technology initiative format. The dataset was divided into training, validation, and testing subsets online test dataset (OTD). The dataset includes four types of MRI series, which were combined together and processed for intensity normalization using contrast limited adaptive histogram equalization methodology. To extract radiomics features, a python-based library called pyRadiomics was employed. Particle-swarm optimization (PSO) with varying inertia weights was used for feature optimization. Inertia weight with a linearly decreasing strategy (W1), inertia weight with a nonlinear coefficient decreasing strategy (W2), and inertia weight with a logarithmic strategy (W3) were different strategies used to vary the inertia weight for feature optimization in PSO. These selected features were further optimized using the principal component analysis (PCA) method to further reducing the dimensionality and removing the noise and improve the performance and efficiency of subsequent algorithms. Support vector machine (SVM), light gradient boosting (LGB), and extreme gradient boosting (XGB) machine learning classification algorithms were utilized for the classification of images into different tumor regions using optimized features. The proposed method was also tested on institute test data (ITD) for a total of 30 patient images. For OTD test dataset, the classification accuracy of SVM was 0.989, for the LGB model (LGBM) was 0.992, and for the XGB model (XGBM) was 0.994, using the varying inertia weight-PSO optimization method and the classification accuracy of SVM was 0.996 for the LGBM was 0.998, and for the XGBM was 0.994, using PSO and PCA-a hybrid optimization technique. For ITD test dataset, the classification accuracy of SVM was 0.994 for the LGBM was 0.993, and for the XGBM was 0.997, using the hybrid optimization technique. The results suggest that the proposed method can be used to classify a brain tumor as used in this study to classify the tumor region into three groups: Peritumoral edema, enhancing-core, and necrotic tumor core. This was done by extracting the different features of the tumor, such as its shape, grey level, gray-level co-occurrence matrix, etc., and then choosing the best features using hybrid optimal feature selection techniques. This was done without much human expertise and in much less time than it would take a person.

MMGan: a multimodal MR brain tumor image segmentation method.

Computer-aided diagnosis has emerged as a rapidly evolving field, garnering increased attention in recent years. At the forefront of this field is the segmentation of lesions in medical images, which is a critical preliminary stage in subsequent treatment procedures. Among the most challenging tasks in medical image analysis is the accurate and automated segmentation of brain tumors in various modalities of brain tumor MRI. In this article, we present a novel end-to-end network architecture called MMGan, which combines the advantages of residual learning and generative adversarial neural networks inspired by classical generative adversarial networks. The segmenter in the MMGan network, which has a U-Net architecture, is constructed using a deep residual network instead of the conventional convolutional neural network. The dataset used for this study is the BRATS dataset from the Brain Tumor Segmentation Challenge at the Medical Image Computing and Computer Assisted Intervention Society. Our proposed method has been extensively tested, and the results indicate that this MMGan framework is more efficient and stable for segmentation tasks. On BRATS 2019, the segmentation algorithm improved accuracy and sensitivity in whole tumor, tumor core, and enhanced tumor segmentation. Particularly noteworthy is the higher dice score of 0.86 achieved by our proposed method in tumor core segmentation, surpassing those of stateof-the-art models. This study improves the accuracy and sensitivity of the tumor segmentation task, which we believe is significant for medical image analysis. And it should be further improved by replacing different loss functions such as cross-entropy loss function and other methods.

Artificial Intelligence-Based Methods for Integrating Local and Global Features for Brain Cancer Imaging: Scoping Review.

Transformer-based models are gaining popularity in medical imaging and cancer imaging applications. Many recent studies have demonstrated the use of transformer-based models for brain cancer imaging applications such as diagnosis and tumor segmentation. This study aims to review how different vision transformers (ViTs) contributed to advancing brain cancer diagnosis and tumor segmentation using brain image data. This study examines the different architectures developed for enhancing the task of brain tumor segmentation. Furthermore, it explores how the ViT-based models augmented the performance of convolutional neural networks for brain cancer imaging. This review performed the study search and study selection following the PRISMA-ScR (Preferred Reporting Items for Systematic Reviews and Meta-Analyses extension for Scoping Reviews) guidelines. The search comprised 4 popular scientific databases: PubMed, Scopus, IEEE Xplore, and Google Scholar. The search terms were formulated to cover the interventions (ie, ViTs) and the target application (ie, brain cancer imaging). The title and abstract for study selection were performed by 2 reviewers independently and validated by a third reviewer. Data extraction was performed by 2 reviewers and validated by a third reviewer. Finally, the data were synthesized using a narrative approach. Of the 736 retrieved studies, 22 (3%) were included in this review. These studies were published in 2021 and 2022. The most commonly addressed task in these studies was tumor segmentation using ViTs. No study reported early detection of brain cancer. Among the different ViT architectures, Shifted Window transformer-based architectures have recently become the most popular choice of the research community. Among the included architectures, UNet transformer and TransUNet had the highest number of parameters and thus needed a cluster of as many as 8 graphics processing units for model training. The brain tumor segmentation challenge data set was the most popular data set used in the included studies. ViT was used in different combinations with convolutional neural networks to capture both the global and local context of the input brain imaging data. It can be argued that the computational complexity of transformer architectures is a bottleneck in advancing the field and enabling clinical transformations. This review provides the current state of knowledge on the topic, and the findings of this review will be helpful for researchers in the field of medical artificial intelligence and its applications in brain cancer.

Multi-class glioma segmentation on real-world data with missing MRI sequences: comparison of three deep learning algorithms

This study tests the generalisability of three Brain Tumor Segmentation (BraTS) challenge models using a multi-center dataset of varying image quality and incomplete MRI datasets. In this retrospective study, DeepMedic, no-new-Unet (nn-Unet), and NVIDIA-net (nv-Net) were trained and tested using manual segmentations from preoperative MRI of glioblastoma (GBM) and low-grade gliomas (LGG) from the BraTS 2021 dataset (1251 in total), in addition to 275 GBM and 205 LGG acquired clinically across 12 hospitals worldwide. Data was split into 80% training, 5% validation, and 15% internal test data. An additional external test-set of 158 GBM and 69 LGG was used to assess generalisability to other hospitals’ data. All models’ median Dice similarity coefficient (DSC) for both test sets were within, or higher than, previously reported human inter-rater agreement (range of 0.74–0.85). For both test sets, nn-Unet achieved the highest DSC (internal = 0.86, external = 0.93) and the lowest Hausdorff distances (10.07, 13.87 mm, respectively) for all tumor classes (p < 0.001). By applying Sparsified training, missing MRI sequences did not statistically affect the performance. nn-Unet achieves accurate segmentations in clinical settings even in the presence of incomplete MRI datasets. This facilitates future clinical adoption of automated glioma segmentation, which could help inform treatment planning and glioma monitoring.

Multiencoder‐based federated intelligent deep learning model for brain tumor segmentation

AbstractGlioma, a primary tumor derived from brain glial cells, is around 45% of all intracranial tumors. Magnetic resonance imaging's (MRI's) precise glioma segmentation is crucial for clinical purposes. This article presents a novel automatic brain tumor segmentation approach based on a multi‐encoder‐based federated intelligent deep learning framework. The suggested method uses a U‐shaped network design that multiplies the single contraction path into several paths to explore semantic information modalities deeply. The basic convolutional layer uses an Inception module and dilated convolutions to extract multi‐scale features from the images using artificial intelligent. To emphasize segmentation‐related information while ignoring redundant channel dimension information and improving the accuracy of network segmentation, lightweight channel attention efficient channel attention (ECA) modules are inserted into the bottleneck layer and decoder. The collection of data for the 2018 Brain Tumor Segmentation Challenge (BraTS 2018) is used to test the effectiveness of the suggested structure, and the findings indicate that the growth core, for the entire tumor and the augmented tumor regions, respectively, the average Dice coefficients are 0.880, 0.784, and 0.757. These findings support the proposed algorithm's ability to accurately and successfully segregate multimodal MRI brain tumors.

Overall Survival Prediction in Stereotactic Radiosurgery Patients with Glioblastoma Via a Deep-Learning Approach

Accurate and automated early survival prediction is critical for glioblastoma (GBM) patients as their poor prognosis requires timely treatment decision-making. We have developed a deep learning (DL)-based GBM overall survival (OS) prediction model based on a multi-institutional public dataset using only pre-operative basic structural multi-parametric magnetic resonance images (MRIs). The purpose of this study is to evaluate this DL-based OS prediction model with an institutional stereotactic radiosurgery (SRS) clinical trial dataset. The task of this study is to classify GBM patients into 3 OS classes: long-survivors (>15 months), mid-survivors (between 10 and 15 months) and short-survivors (< 10 months). The proposed OS prediction model is an ensemble of a ResNet-based classifier and a K-NN classifier. The ResNet-based classifier is trained in a Siamese fashion to explore inter-class differences. During testing, training sample features are implemented with a K-NN classifier to ensemble with the ResNet-based classifier. A public dataset from Medical Image Computing and Computer Assisted Intervention (MICCAI) Brain Tumor Segmentation (BraTS) challenge 2020 (235 patients) were used for model establishing and initial validation. Then the validated model was evaluated on 19 GBM patients from an institutional SRS clinical trial. Each data entry consists of pre-operative basic structural multi-parametric MRIs and survival days, as well as patient ages for BraTS data and basic clinical characteristics for institutional data. GBM sub-regions, including contrast-enhancing tumor, peri-tumoral edema, and necrotic/non-enhancing tumor core, were segmented in the multi-parametric MRIs by an in-house DL model for both datasets. The OS prediction model was trained on 90% of the segmented BraTS data and validated on the rest 10%, then further evaluated on the institutional data. The model performance was assessed by prediction accuracy (ACC) and the area under the curve (AUC). For this 3-class OS classification task, our DL-based prediction model achieved an ACC of 65.22% and an AUC of 0.81 on the BraTS dataset compared with the top-ranked result from the BraTS challenge 2020 (Rank 1st: ACC 61.7%), and an ACC of 52.63% and an AUC of 0.69 on the institutional dataset. Further analysis of the institutional dataset found that the predicted OS class had a statistically significant correlation with treatment volume (p = 0.012) and age (p = 0.006), which matches the analysis that the patients' ground truth OS class is statistical significantly correlated with treatment volume (p = 0.045). Our DL-based OS prediction model for GBM using basic structural multi-parametric pre-operative MRIs has demonstrated promising performance in both public and institutional dataset with minimal manual processing requirements. This OS prediction model can be potentially applied to assist timely clinical decision-making.

An active learning approach to train a deep learning algorithm for tumor segmentation from brain MR images

PurposeThis study focuses on assessing the performance of active learning techniques to train a brain MRI glioma segmentation model.MethodsThe publicly available training dataset provided for the 2021 RSNA-ASNR-MICCAI Brain Tumor Segmentation (BraTS) Challenge was used in this study, consisting of 1251 multi-institutional, multi-parametric MR images. Post-contrast T1, T2, and T2 FLAIR images as well as ground truth manual segmentation were used as input for the model. The data were split into a training set of 1151 cases and testing set of 100 cases, with the testing set remaining constant throughout. Deep convolutional neural network segmentation models were trained using the NiftyNet platform. To test the viability of active learning in training a segmentation model, an initial reference model was trained using all 1151 training cases followed by two additional models using only 575 cases and 100 cases. The resulting predicted segmentations of these two additional models on the remaining training cases were then addended to the training dataset for additional training.ResultsIt was demonstrated that an active learning approach for manual segmentation can lead to comparable model performance for segmentation of brain gliomas (0.906 reference Dice score vs 0.868 active learning Dice score) while only requiring manual annotation for 28.6% of the data.ConclusionThe active learning approach when applied to model training can drastically reduce the time and labor spent on preparation of ground truth training data.Critical relevance statementActive learning concepts were applied to a deep learning-assisted segmentation of brain gliomas from MR images to assess their viability in reducing the required amount of manually annotated ground truth data in model training.Key points• This study focuses on assessing the performance of active learning techniques to train a brain MRI glioma segmentation model.• The active learning approach for manual segmentation can lead to comparable model performance for segmentation of brain gliomas.• Active learning when applied to model training can drastically reduce the time and labor spent on preparation of ground truth training data.Graphical

Brain Tumor Segmentation from MRI Images Using Handcrafted Convolutional Neural Network.

Brain tumor segmentation from magnetic resonance imaging (MRI) scans is critical for the diagnosis, treatment planning, and monitoring of therapeutic outcomes. Thus, this research introduces a novel hybrid approach that combines handcrafted features with convolutional neural networks (CNNs) to enhance the performance of brain tumor segmentation. In this study, handcrafted features were extracted from MRI scans that included intensity-based, texture-based, and shape-based features. In parallel, a unique CNN architecture was developed and trained to detect the features from the data automatically. The proposed hybrid method was combined with the handcrafted features and the features identified by CNN in different pathways to a new CNN. In this study, the Brain Tumor Segmentation (BraTS) challenge dataset was used to measure the performance using a variety of assessment measures, for instance, segmentation accuracy, dice score, sensitivity, and specificity. The achieved results showed that our proposed approach outperformed the traditional handcrafted feature-based and individual CNN-based methods used for brain tumor segmentation. In addition, the incorporation of handcrafted features enhanced the performance of CNN, yielding a more robust and generalizable solution. This research has significant potential for real-world clinical applications where precise and efficient brain tumor segmentation is essential. Future research directions include investigating alternative feature fusion techniques and incorporating additional imaging modalities to further improve the proposed method's performance.