Impact of Synthetic Contrast-Enhanced MRI on Brain Tumor Segmentation

The application of deep learning in the field of medical imaging has become increasingly widespread, greatly promoting the advancement and development of Magnetic Resonance Imaging (MRI) brain tumor detection and segmentation techniques. Therefore, a comprehensive review of deep learning-based methods for MRI brain tumor detection and segmentation was conducted. This review introduces the basic concepts of brain tumors and MRI brain tumor detection and segmentation, discusses the specific applications and typical methods of deep learning in MRI brain tumor detection and segmentation, and analyzes and compares the performance and advantages and disadvantages of different methods. Additionally, representative brain tu-mor segmentation dataset (BraTS) and its evaluation metrics are introduced, upon which the performance of various deep learning-based brain tumor segmentation methods on the BraTS 2019-2022 dataset is compared. Lastly, the challenges and future development trends in deep learning-based MRI brain tumor detection and segmentation methods are summarized and anticipated.

PurposeThe purpose of this study is to develop accurate and automated detection and segmentation methods for brain tumors, given their significant fatality rates, with aggressive malignant tumors like Glioblastoma Multiforme (GBM) having a five-year survival rate as low as 5 to 10%. This underscores the urgent need to improve diagnosis and treatment outcomes through innovative approaches in medical imaging and deep learning techniques.MethodsIn this work, we propose a novel approach utilizing the two-headed UNetEfficientNets model for simultaneous segmentation and classification of brain tumors from Magnetic Resonance Imaging (MRI) images. The model combines the strengths of EfficientNets and a modified two-headed Unet model. We utilized a publicly available dataset consisting of 3064 brain MR images classified into three tumor classes: Meningioma, Glioma, and Pituitary. To enhance the training process, we performed 12 types of data augmentation on the training dataset. We evaluated the methodology using six deep learning models, ranging from UNetEfficientNet-B0 to UNetEfficientNet-B5, optimizing the segmentation and classification heads using binary cross entropy (BCE) loss with Dice and BCE with focal loss, respectively. Post-processing techniques such as connected component labeling (CCL) and ensemble models were applied to improve segmentation outcomes.ResultsThe proposed UNetEfficientNet-B4 model achieved outstanding results, with an accuracy of 99.4% after postprocessing. Additionally, it obtained high scores for DICE (94.03%), precision (98.67%), and recall (99.00%) after post-processing. The ensemble technique further improved segmentation performance, with a global DICE score of 95.70% and Jaccard index of 91.20%.ConclusionOur study demonstrates the high efficiency and accuracy of the proposed UNetEfficientNet-B4 model in the automatic and parallel detection and segmentation of brain tumors from MRI images. This approach holds promise for improving diagnosis and treatment planning for patients with brain tumors, potentially leading to better outcomes and prognosis.Supplementary InformationThe online version contains supplementary material available at 10.1007/s00432-024-05718-1.

Deep learning shows promise in automated brain tumour segmentation, but it depends on costly expert annotations. Recent advances in unsupervised learning offer an alternative by using synthetic data for training. However, the discrepancy between real and synthetic data limits the accuracy of the unsupervised approaches. In this paper, we propose an approach for unsupervised brain tumour segmentation on magnetic resonance (MR) images via a two-stage image synthesis strategy. This approach accounts for the domain gap between real and synthetic data and aims to generate realistic synthetic data for model training. In the first stage, we train a junior segmentation model using synthetic brain tumour images generated by hand-crafted tumour shape and intensity models, and employs a validation set with distribution shift for model selection. The trained junior model is applied to segment unlabelled real tumour images, generating pseudo labels that capture realistic tumour shape, intensity, and texture. In the second stage, realistic synthetic tumour images are generated by mixing brain images with tumour pseudo labels, closing the domain gap between real and synthetic images. The generated synthetic data is then used to train a senior model for final segmentation. In experiments on five brain imaging datasets, the proposed approach, named as SynthTumour, surpasses existing unsupervised methods and demonstrates high performance for both brain tumour segmentation and ischemic stroke lesion segmentation tasks.

The accurate recognition of a brain tumor (BT) is crucial for accurate diagnosis, intervention planning, and the evaluation of post-intervention outcomes. Conventional methods of manually identifying and delineating BTs are inefficient, prone to error, and time-consuming. Subjective methods for BT recognition are biased because of the diffuse and irregular nature of BTs, along with varying enhancement patterns and the coexistence of different tumor components. Hence, the development of an automated diagnostic system for BTs is vital for mitigating subjective bias and achieving speedy and effective BT segmentation. Recently developed deep learning (DL)-based methods have replaced subjective methods; however, these DL-based methods still have a low performance, showing room for improvement, and are limited to heterogeneous dataset analysis. Herein, we propose a DL-based parallel features aggregation network (PFA-Net) for the robust segmentation of three different regions in a BT scan, and we perform a heterogeneous dataset analysis to validate its generality. The parallel features aggregation (PFA) module exploits the local radiomic contextual spatial features of BTs at low, intermediate, and high levels for different types of tumors and aggregates them in a parallel fashion. To enhance the diagnostic capabilities of the proposed segmentation framework, we introduced the fractal dimension estimation into our system, seamlessly combined as an end-to-end task to gain insights into the complexity and irregularity of structures, thereby characterizing the intricate morphology of BTs. The proposed PFA-Net achieves the Dice scores (DSs) of 87.54%, 93.42%, and 91.02%, for the enhancing tumor region, whole tumor region, and tumor core region, respectively, with the multimodal brain tumor segmentation (BraTS)-2020 open database, surpassing the performance of existing state-of-the-art methods. Additionally, PFA-Net is validated with another open database of brain tumor progression and achieves a DS of 64.58% for heterogeneous dataset analysis, surpassing the performance of existing state-of-the-art methods.

Gliomas are very heterogenous set of tumors that grow within the substance of brain and often mix with normal brain tissues. Due to its histologic complexity and irregular shapes, multiparametric magnetic resonance imaging is used to accurately diagnose brain tumor and their subregions. Current practice requires physicians to manually segment these regions on a large image dataset, which can be a very time consuming and complicated task especially with large variations among different tumor regions. Automatic segmentation of brain tumors in multimodal MRI holds a great potential in developing an effective treatment plan and improving brain tumor radiotherapy workflow. Despite continuous investigations on DL-based brain tumor segmentation, irregular shapes and histologic complexities of brain tumors introduces a major challenge in developing an effective automatic segmentation method. In this study, we develop a novel context U-net with deep supervision to segment both the whole brain tumor and their subregions of tumors. The context module was formed by an inception-like structure to extract more information regarding the brain tumors. The deep supervision in the encoder path was achieved by adding up the segmentation outputs at different levels of the network. We evaluated our method on Brain Tumor Segmentation Challenge (BraTS) 2019 training dataset in which 80% was used for training while the remaining 20% was used for performance testing. Our method achieved Dice similarity coefficients (DSC) were 0.8693, 0.8013 and 0.7782 for the whole tumor (WT), tumor core (TC) and enhancing tumor (ET), respectively. The results attained by our proposed network suggested that this technique could be used for segmentation of brain tumors and their subregions to facilitate the brain tumor radiotherapy workflow.

Accurate segmentation of different brain tumor regions from MR images is of great significance in the diagnosis and treatment of brain tumors. In this paper, an enhanced 3D U-Net model was proposed to address the shortcomings of 2D U-Net in the segmentation tasks of brain tumors. While retaining the U-shaped characteristics of the original U-Net network, an enhanced encoding module and decoding module were designed to increase the extraction and utilization of image features. Then, a hybrid loss function combining the binary cross-entropy loss function and dice similarity coefficient was adopted to speed up the model’s convergence and to achieve accurate and fast automatic segmentation. The model’s performance in the segmentation of brain tumor’s whole tumor region, tumor core region, and enhanced tumor region was studied. The results showed that the proposed 3D U-Net model can achieve better segmentation performance, especially for the tumor core region and enhanced tumor region tumor regions.

Deep multi-task learning structure for segmentation and classification of supratentorial brain tumors in MR images

A systematic review on deep learning based brain tumor segmentation and detection using MRI: Past insights, present techniques and future trends.

A Validation Framework for Brain Tumor Segmentation

Brain tumor is an anomalous growth of brain cells. Segmentation of brain tumors is currently the most important surgical and pharmaceutical procedure. However, manually segmenting the brain tumor is a challenging task due to the complex structure of brain. In recent years, artificial intelligence techniques with the fuzzy logic have shown better results in the field of medicine. In this work, a novel deep learning classification network with fuzzy hexagonal membership function (DLC-FHMF) model has been proposed for accurately segmenting brain tumors. The different MRI modalities namely T1, T1-c, T2 and Flair images are preprocessed using a fuzzy hexagonal trilateral and median filter to eliminate the Rician noise. Afterwards, the DLC-FHMF model is used for segmenting the tumor portion by using the multimodal composition of MRI as input. The fuzzy weights are determined with hexagonal membership functions and convoluted with the corresponding MRI images. The quantitative examination is carried out using the performance metrics namely accuracy, specificity, precision, sensitivity, incorrect segmentation, under-segmentation, and over-segmentation. In addition to the above metrics, the pre-processing metrics include PSNR, RMSE, and SSIM. The experimental fallout portrayals that the proposed DLC-FHMF approach attains a better accuracy range of 99% for detecting brain tumors using the BRATS 2013 dataset. The proposed DLC-FHMF model improves the overall accuracy by 15.1%, 11.1%, 3.0%, 21.2% and 0.5% better than ANN, SVM, NB, DNN and DAE respectively.

Accurate brain tumor segmentation using multi-sequence magnetic resonance (MR) imaging plays a pivotal role in clinical practice and research settings. Despite their prevalence, deep learning-based segmentation methods, which usually use multiple MR sequences as input, still have limited performance, partly due to their insufficient ability to image representation. In this paper, we propose a brain tumor segmentation (BraTSeg) model, which uses cross-sequence MR image generation as a self-supervision tool to improve the segmentation accuracy. This model is an ensemble of three image segmentation and generation (ImgSG) models, which are designed for simultaneous segmentation of brain tumors and generation of T1, T2, and Flair sequences, respectively. We evaluated the proposed BraTSeg model on the BraTS 2019 dataset and achieved an average Dice similarity coefficient (DSC) of 81.93%, 87.80%, and 83.44% in the segmentation of enhancing tumor, whole tumor, and tumor score on the testing set, respectively. Our results suggest that using cross-sequence MR image generation is an effective self-supervision method that can improve the accuracy of brain tumor segmentation and the proposed BraTSeg model can produce satisfactory segmentation of brain tumors and intra-tumor structures.KeywordsBrain tumor segmentationDeep learningMulti-sequence MR imagingMR image generation

Segmentation of tumor from magnetic resonance image (MRI) brain images is an emergent research area in the field of medical image segmentation. As segmentation of brain tumor plays an important role for necessary treatment and planning of tumor surgery. However, segmentation of the brain tumor is still a great challenge in clinics, specially automatic segmentation. In this paper we proposed hybridized multilevel thresholding and level set method for automatic segmentation of brain tumor. The innovation for this paper is to interface the initial segmentation from multilevel thresholding and extract a fine portrait using level set method with morphological operations. The results are compared with the existing method and also with radiologist manual segmentation which confirm the effectiveness of this hybridized paradigm for brain tumor segmentation.

Background: Accurate detection of brain tumor and its severity is a challenging task in the medical field. So there is a need for developing brain tumor detecting algorithms and it is an emerging one for diagnosis, planning the treatment and outcome evaluation. Materials and Methods: Brain tumor segmentation method using deep learning classification and multi-modal composition has been developed using the deep convolutional neural networks. The different modalities of MRI such as T1, flair, T1C and T2 are given as input for the proposed method. The MR images from the different modalities are used in proportion to the information contents in the particular modality. The weights for the different modalities are calculated blockwise and the standard deviation of the block is taken as a proxy for the information content of the block. Then the convolution is performed between the input image of the T1, flair, T1C and T2 MR images and corresponding to the weight of the T1, flair, T1C, and T2 images. The convolution is summed between the different modalities of the MR images and its corresponding weight of the different modalities of the MR images to obtain a new composite image which is given as an input image to the deep convolutional neural network. The deep convolutional neural network performs segmentation through the different layers of CNN and different filter operations are performed in each layer to obtain the enhanced classification and segmented spatial consistency results. The analysis of the proposed method shows that the discriminatory information from the different modalities is effectively combined to increase the overall accuracy of segmentation. Results: The proposed deep convolutional neural network for brain tumor segmentation method has been analysed by using the Brain Tumor Segmentation Challenge 2013 database (BRATS 2013). The complete, core and enhancing regions are validated with Dice Similarity Coefficient and Jaccard similarity index metric for the Challenge, Leaderboard, and Synthetic data set. To evaluate the classification rates, the metrics such as accuracy, precision, sensitivity, specificity, under-segmentation, incorrect segmentation and over segmentation also evaluated and compared with the existing methods. Experimental results exhibit a higher degree of precision in the segmentation compared to existing methods. Conclusion: In this work, deep convolution neural network with different modalities of MR image are used to detect the brain tumor. The new input image was created by convoluting the input image of the different modalities and their weights. The weights are determined using the standard deviation of the block. Segmentation accuracy is high with efficient appearance and spatial consistency. The assessment of segmented images is completely evaluated by using well-established metrics. In future, the proposed method will be considered and evaluated with other databases and the segmentation accuracy results should be analysed with the presence of different kind of noises.

Chapter 11 - Deep Learning for Automated Brain Tumor Segmentation in MRI Images

Accurate tumor segmentation has the ability to provide doctors with a basis for surgical planning. Moreover, brain tumor segmentation needs to extract different tumor tissues (Edema, tumor, tumor enhancement, and necrosis) from normal tissues which is a big challenge because tumor structures vary considerably across patients in terms of size, extension, and localization. In this article, we evaluate a fully automated method for segmenting brain tumor images from multi‐modal magnetic resonance imaging volumes based on stacked de‐noising auto‐encoders (SDAEs). Specially, we adopted multi‐modality information from T1, T1c, T2, and Flair images, respectively. We extracted gray level patches from different modalities as the input of the SDAE. After trained by the SDAE, the raw network parameters will be obtained, which are adopted as a parameter of the feed forward neural network for classification. A simple post‐processing is implemented by threshold segmentation method to generate a mask to get the final segmentation result. By evaluating the proposed method on the BRATS 2015, it can be proven that our method obtains the better performance than other state‐of‐the‐art counterpart methods. And a preliminary dice score of 0.86 for whole tumor segmentation has been achieved.