Multimodal Brain Tumor Segmentation Research Articles

Uncertainty quantification and attention-aware fusion guided multi-modal MR brain tumor segmentation

Brain tumor is one of the most aggressive cancers in the world, accurate brain tumor segmentation plays a critical role in clinical diagnosis and treatment planning. Although deep learning models have presented remarkable success in medical segmentation, they can only obtain the segmentation map without capturing the segmentation uncertainty. To achieve accurate and safe clinical results, it is necessary to produce extra uncertainty maps to assist the subsequent segmentation revision. To this end, we propose to exploit the uncertainty quantification in the deep learning model and apply it to multi-modal brain tumor segmentation. In addition, we develop an effective attention-aware multi-modal fusion method to learn the complimentary feature information from the multiple MR modalities. First, a multi-encoder-based 3D U-Net is proposed to obtain the initial segmentation results. Then, an estimated Bayesian model is presented to measure the uncertainty of the initial segmentation results. Finally, the obtained uncertainty maps are integrated into a deep learning-based segmentation network, serving as an additional constraint information to further refine the segmentation results. The proposed network is evaluated on publicly available BraTS 2018 and BraTS 2019 datasets. The experimental results demonstrate that the proposed method outperforms the previous state-of-the-art methods on Dice score, Hausdorff distance and Sensitivity metrics. Furthermore, the proposed components could be easily applied to other network architectures and other computer vision fields.

Prediction of brain tumor recurrence location based on multi-modal fusion and nonlinear correlation learning.

Brain tumor is one of the leading causes of cancer death. The high-grade brain tumors are easier to recurrent even after standard treatment. Therefore, developing a method to predict brain tumor recurrence location plays an important role in the treatment planning and it can potentially prolong patient's survival time. There is still little work to deal with this issue. In this paper, we present a deep learning-based brain tumor recurrence location prediction network. Since the dataset is usually small, we propose to use transfer learning to improve the prediction. We first train a multi-modal brain tumor segmentation network on the public dataset BraTS 2021. Then, the pre-trained encoder is transferred to our private dataset for extracting the rich semantic features. Following that, a multi-scale multi-channel feature fusion model and a nonlinear correlation learning module are developed to learn the effective features. The correlation between multi-channel features is modeled by a nonlinear equation. To measure the similarity between the distributions of original features of one modality and the estimated correlated features of another modality, we propose to use Kullback-Leibler divergence. Based on this divergence, a correlation loss function is designed to maximize the similarity between the two feature distributions. Finally, two decoders are constructed to jointly segment the present brain tumor and predict its future tumor recurrence location. To the best of our knowledge, this is the first work that can segment the present tumor and at the same time predict future tumor recurrence location, making the treatment planning more efficient and precise. The experimental results demonstrated the effectiveness of our proposed method to predict the brain tumor recurrence location from the limited dataset.

Attention-guided multi-scale context aggregation network for multi-modal brain glioma segmentation.

Accurate segmentation of brain glioma is a critical prerequisite for clinical diagnosis, surgical planning and treatment evaluation. In current clinical workflow, physicians typically perform delineation of brain tumor subregions slice-by-slice, which is more susceptible to variabilities in raters and also time-consuming. Besides, even though convolutional neural networks (CNNs) are driving progress, the performance of standard models still have some room for furtherimprovement. To deal with these issues, this paper proposes an attention-guided multi-scale context aggregation network (AMCA-Net) for the accurate segmentation of brain glioma in the magnetic resonance imaging (MRI) images withmulti-modalities. AMCA-Net extracts the multi-scale features from the MRI images and fuses the extracted discriminative features via a self-attention mechanism for brain glioma segmentation. The extraction is performed via a series of down-sampling, convolution layers, and the global context information guidance (GCIG) modules are developed to fuse the features extracted for contextual features. At the end of the down-sampling, a multi-scale fusion (MSF) module is designed to exploit and combine all the extracted multi-scale features. Each of the GCIG and MSF modules contain a channel attention (CA) module that can adaptively calibrate feature responses and emphasize the most relevant features. Finally, multiple predictions with different resolutions are fused through different weightings given by a multi-resolution adaptation (MRA) module instead of the use of averaging or max-pooling to improve the final segmentationresults. Datasets used in this paper are publicly accessible, that is, the Multimodal Brain Tumor Segmentation Challenges 2018 (BraTS2018) and 2019 (BraTS2019). BraTS2018 contains 285 patient cases and BraTS2019 contains 335 cases. Simulations show that the AMCA-Net has better or comparable performance against that of the other state-of-the-art models. In terms of the Dice score and Hausdorff 95 for the BraTS2018 dataset, 90.4% and 10.2 mm for the whole tumor region (WT), 83.9% and 7.4 mm for the tumor core region (TC), 80.2% and 4.3 mm for the enhancing tumor region (ET), whereas the Dice score and Hausdorff 95 for the BraTS2019 dataset, 91.0% and 10.7 mm for the WT, 84.2% and 8.4 mm for the TC, 80.1% and 4.8 mm for the ET. The proposed AMCA-Net performs comparably well in comparison to several state-of-the-art neural net models in identifying the areas involving the peritumoral edema, enhancing tumor, and necrotic and non-enhancing tumor core of brain glioma, which has great potential for clinical practice. In future research, we will further explore the feasibility of applying AMCA-Net to other similar segmentationtasks.

Feature fusion and latent feature learning guided brain tumor segmentation and missing modality recovery network

Accurate brain tumor segmentation is an essential step for clinical diagnosis and surgical treatment. Multimodal brain tumor segmentation strongly relies on an effective fusion method and an excellent segmentation network. However, it is common to have some missing MR modalities in clinical scenarios due to image corruption, acquisition protocol, scanner availability and scanning cost, which can heavily decrease the tumor segmentation accuracy, and also cause information loss for down-streaming disease analysis. To address this issue, I propose a novel multimodal feature fusion and latent feature learning guided deep neural network. On the one hand, the proposed network can help to segment brain tumors when one or more modalities are missing. On the other hand, it can retrieve the missing modalities to compensate for incomplete data. The proposed network consists of three key components. First, a Multimodal Feature Fusion Module (MFFM) is proposed to effectively fuse the complementary information from different modalities, consisting of a Cross-Modality Fusion Module (CMFM) and a Multi-Scale Fusion Module (MSFM). Second, a Spatial Consistency-based Latent Feature Learning Module (SC-LFLM) is presented to exploit multimodal latent correlation and extract the relevant features to benefit segmentation. Third, the Multi-Task Learning (MTL) paths are integrated to supervise the segmentation and recover the missing modalities. The proposed method is evaluated on BraTS 2018 dataset, and it can achieve superior segmentation results when one or more modalities are missing, compared with the state-of-the-art methods. Furthermore, the proposed modules can be easily adapted to other multimodal network architectures and research fields.

An improved DNN with FFCM method for multimodal brain tumor segmentation

A brain tumor is one of the deadliest neurological diseases developed in the human brain. Gliomas are the most common type of brain tumor which are originated from the glial cells of the brain and are treated as the most treacherous tumors due to aggressiveness and heterogeneous structure. Early detection of gliomas increases the survival rate of the patients. In this paper, a hybrid deep neural network is proposed that uses inception v2 network hybridized with 16 new layered segmentation nets. The network is tested on BraTs 2020 and BraTs 2017 multi-parametric MRI (mPMRI) dataset to detect the whole tumor, and for the detection of tumor core (TC) and the edema, fast fuzzy C-means (FFCM) method is used. The proposed method attains an accuracy of 99.45% and a Dice similarity coefficient (DSC) of 89.74% for the whole tumor, an accuracy of 99.36%, a DSC of 86.47% for the tumor core and accuracy of 99.45%, and a DSC of 85.22% for the edema.

Learning multi-modal brain tumor segmentation from privileged semi-paired MRI images with curriculum disentanglement learning

Since the brain is the human body’s primary command and control center, brain cancer is one of the most dangerous cancers. Automatic segmentation of brain tumors from multi-modal images is important in diagnosis and treatment. Due to the difficulties in obtaining multi-modal paired images in clinical practice, recent studies segment brain tumors solely relying on unpaired images and discarding the available paired images. Although these models solve the dependence on paired images, they cannot fully exploit the complementary information from different modalities, resulting in low unimodal segmentation accuracy. Hence, this work studies the unimodal segmentation with privileged semi-paired images, i.e., limited paired images are introduced to the training phase. Specifically, we present a novel two-step (intra-modality and inter-modality) curriculum disentanglement learning framework. The modality-specific style codes describe the attenuation of tissue features and image contrast, and modality-invariant content codes contain anatomical and functional information extracted from the input images. Besides, we address the problem of unthorough decoupling by introducing constraints on the style and content spaces. Experiments on the BraTS2020 dataset highlight that our model outperforms the competing models on unimodal segmentation, achieving average dice scores of 82.91%, 72.62%, and 54.80% for WT (the whole tumor), TC (the tumor core), and ET (the enhancing tumor), respectively. Finally, we further evaluate our model’s variable multi-modal brain tumor segmentation performance by introducing a fusion block (TFusion). The experimental results reveal that our model achieves the best WT segmentation performance for all 15 possible modality combinations with 87.31% average accuracy. In summary, we propose a curriculum disentanglement learning framework for unimodal segmentation with privileged semi-paired images. Moreover, the benefits of the improved unimodal segmentation extend to variable multi-modal segmentation, demonstrating that improving the unimodal segmentation performance is significant for brain tumor segmentation with missing modalities. Our code is available at https://github.com/scut-cszcl/SpBTS.

Batch size: go big or go home? Counterintuitive improvement in medical autoencoders with smaller batch size.

Batch size is a key hyperparameter in training deep learning models. Conventional wisdom suggests larger batches produce improved model performance. Here we present evidence to the contrary, particularly when using autoencoders to derive meaningful latent spaces from data with spatially global similarities and local differences, such as electronic health records (EHR) and medical imaging. We investigate batch size effects in both EHR data from the Baltimore Longitudinal Study of Aging and medical imaging data from the multimodal brain tumor segmentation (BraTS) challenge. We train fully connected and convolutional autoencoders to compress the EHR and imaging input spaces, respectively, into 32-dimensional latent spaces via reconstruction losses for various batch sizes between 1 and 100. Under the same hyperparameter configurations, smaller batches improve loss performance for both datasets. Additionally, latent spaces derived by autoencoders with smaller batches capture more biologically meaningful information. Qualitatively, we visualize 2-dimensional projections of the latent spaces and find that with smaller batches the EHR network better separates the sex of the individuals, and the imaging network better captures the right-left laterality of tumors. Quantitatively, the analogous sex classification and laterality regressions using the latent spaces demonstrate statistically significant improvements in performance at smaller batch sizes. Finally, we find improved individual variation locally in visualizations of representative data reconstructions at lower batch sizes. Taken together, these results suggest that smaller batch sizes should be considered when designing autoencoders to extract meaningful latent spaces among EHR and medical imaging data driven by global similarities and local variation.

Attention-based multimodal glioma segmentation with multi-attention layers for small-intensity dissimilarity

The segmentation of glioma by computer vision is one of the hot topics in medical image analysis, which further helps doctors to make a better treatment plan for glioma. At present, convolutional neural networks (CNN) with multi-kernels are the mainstream method to identify glioma regions. However, the segmentation is strongly affected if the intensity dissimilarity between adjacent glioma regions is small. To solve this challenge, we propose an attention-based multimodal glioma segmentation with multi-attention layers for small-intensity dissimilarity to focus on the small-intensity dissimilarity glioma regions. Firstly, to reduce background interferences, this paper proposes data enhancement in glioma-centered regions. In addition, a random multi-dimensional data view is generated in the glioma regions to reduce overfitting. Secondly, we embed a 3D U-Net to proposed attention layers, which focus on the intensity dissimilarity between adjacent glioma regions, and adaptively mine the glioma-related features, solving the problem that the existing algorithms are insensitive to the small-intensity dissimilarity between adjacent glioma regions. In particular, each attention layer can adaptively highlight valuable glioma features and suppress unrelated ones. Finally, experiment results on the multimodal brain tumor segmentation challenge (BraTS) 2020 dataset validate the effectiveness of the proposed method, where the Dice Similarity Coefficients (DSC) are improved on the segmentation of whole tumor (WT), tumor core (TC), and enhanced tumor (ET) regions, reaching higher results at 0.7803, 0.8831 and 0.8172 for WT, TC and ET regions respectively. Also, we make a test on the public dataset BraTS2019, reaching the results at 0.7675, 0.8925, and 0.8110 for WT, TC, and ET regions, respectively.

A literature survey of MR-based brain tumor segmentation with missing modalities.

Multimodal MR brain tumor segmentation is one of the hottest issues in the community of medical image processing. However, acquiring the complete set of MR modalities is not always possible in clinical practice, due to the acquisition protocols, image corruption, scanner availability, scanning cost or allergies to certain contrast materials. The missing information can cause some restraints to brain tumor diagnosis, monitoring, treatment planning and prognosis. Thus, it is highly desirable to develop brain tumor segmentation methods to address the missing modalities problem. Based on the recent advancements, in this review, we provide a detailed analysis of the missing modality issue in MR-based brain tumor segmentation. First, we briefly introduce the biomedical background concerning brain tumor, MR imaging techniques, and the current challenges in brain tumor segmentation. Then, we provide a taxonomy of the state-of-the-art methods with five categories, namely, image synthesis-based method, latent feature space-based model, multi-source correlation-based method, knowledge distillation-based method, and domain adaptation-based method. In addition, the principles, architectures, benefits and limitations are elaborated in each method. Following that, the corresponding datasets and widely used evaluation metrics are described. Finally, we analyze the current challenges and provide a prospect for future development trends. This review aims to provide readers with a thorough knowledge of the recent contributions in the field of brain tumor segmentation with missing modalities and suggest potential future directions.

Inner Cascaded U2-Net: An Improvement to Plain Cascaded U-Net

Deep neural networks are now widely used in the medical image segmentation field for their performance superiority and no need of manual feature extraction. U-Net has been the baseline model since the very beginning due to a symmetrical U-structure for better feature extraction and fusing and suitable for small datasets. To enhance the segmentation performance of U-Net, cascaded U-Net proposes to put two U-Nets successively to segment targets from coarse to fine. However, the plain cascaded U-Net faces the problem of too less between connections so the contextual information learned by the former U-Net cannot be fully used by the latter one. In this article, we devise novel Inner Cascaded U-Net and Inner Cascaded U2-Net as improvements to plain cascaded U-Net for medical image segmentation. The proposed Inner Cascaded U-Net adds inner nested connections between two U-Nets to share more contextual information. To further boost segmentation performance, we propose Inner Cascaded U2-Net, which applies residual U-block to capture more global contextual information from different scales. The proposed models can be trained from scratch in an end-to-end fashion and have been evaluated on Multimodal Brain Tumor Segmentation Challenge (BraTS) 2013 and ISBI Liver Tumor Segmentation Challenge (LiTS) dataset in comparison to related U-Net, cascaded U-Net, U-Net++, U2-Net and state-of-the-art methods. Our experiments demonstrate that our proposed Inner Cascaded U-Net and Inner Cascaded U2-Net achieve better segmentation performance in terms of dice similarity coefficient and hausdorff distance as well as get finer outline segmentation.

Adaptive Cross-Feature Fusion Network With Inconsistency Guidance for Multi-Modal Brain Tumor Segmentation.

In the context of contemporary artificial intelligence, increasing deep learning (DL) based segmentation methods have been recently proposed for brain tumor segmentation (BraTS) via analysis of multi-modal MRI. However, known DL-based works usually directly fuse the information of different modalities at multiple stages without considering the gap between modalities, leaving much room for performance improvement. In this paper, we introduce a novel deep neural network, termed ACFNet, for accurately segmenting brain tumor in multi-modal MRI. Specifically, ACFNet has a parallel structure with three encoder-decoder streams. The upper and lower streams generate coarse predictions from individual modality, while the middle stream integrates the complementary knowledge of different modalities and bridges the gap between them to yield fine prediction. To effectively integrate the complementary information, we propose an adaptive cross-feature fusion (ACF) module at the encoder that first explores the correlation information between the feature representations from upper and lower streams and then refines the fused correlation information. To bridge the gap between the information from multi-modal data, we propose a prediction inconsistency guidance (PIG) module at the decoder that helps the network focus more on error-prone regions through a guidance strategy when incorporating the features from the encoder. The guidance is obtained by calculating the prediction inconsistency between upper and lower streams and highlights the gap between multi-modal data. Extensive experiments on the BraTS 2020 dataset show that ACFNet is competent for the BraTS task with promising results and outperforms six mainstream competing methods.

Comparative Analysis of Deep Learning Models on Brain Tumor Segmentation Datasets: BraTS 2015-2020 Datasets

Deep Learning neural networks have shown applicability in segmentation of brain tumor images.This research have been carried for comprehensive review of several deep learning neural networks. The datasets included in this study are standard datasets Multimodal Brain Tumor Segmentation (BraTS). This paper has summarized the performance of various deep learning neural network algorithms on BraTS datasets. Algorithms have been compared and summarized against the baseline models with specific attributes like dice score, PPV and sensitivity. It has been found that out of the different models applied on the BraTS 2015 dataset GAN in the year 2020 algorithm is showing better results on this data set. GAN architecture termed RescueNet gave the best segmentation results in terms of 0.94 dice score and 0.88 Sensitivity. This has been also observed that models used cascaded deep learning models had independent deep learning models at each stage which had no correlation among the stages which can cause class imbalance. Further it have found that the Attention models tried to solve problem of class imbalance in the brain tumor segmentation task. This work also found that existing CNN’s is having overfitting issues. For this ResNet models can add a rapid connect bounce relationship parallel to the layers of CNN to accomplish better outcomes for the brain tumor segmentation task.

Modality-level cross-connection and attentional feature fusion based deep neural network for multi-modal brain tumor segmentation

Brain tumor segmentation from Magnetic Resonance Imaging is essential for early diagnosis and treatment planning for brain cancers in clinical practice. However, existing brain tumor segmentation methods cannot sufficiently learn high-quality feature information for segmentation. To address this issue, a modality-level cross-connection and attentional feature fusion based deep neural network is proposed for multi-modal brain tumor segmentation. The proposed method can not only locate the whole tumor region but also can accurately segment the sub-tumor regions. The proposed network architecture is a multi-encoder based 3D U-Net. Inspired by the characteristics of multi-modalities, a modality-level cross-connection (MCC) is first proposed to take advantage of the complementary information between the related modalities. Moreover, to enhance the feature learning capacity of the network, the attentional feature fusion module (AFFM) is proposed to fuse the multi-modalities as well as to extract the useful feature representation for segmentation. It consists of two components: multi-scale spatial feature fusion (MSFF) block and dual-path channel feature fusion (DCFF) block. They aim at learning multi-scale spatial contextual information and the channel-wise feature information to improve the segmentation accuracy. Also, the proposed fusion module can be easily integrated into other fusion models and deep neural network architectures. Comprehensive experiments evaluated on the BraTS 2018 dataset demonstrate that the proposed network architecture can effectively improve the brain tumor segmentation performance when compared with the baseline methods and the state-of-the-art methods.

DeepEOR: automated perioperative volumetric assessment of variable grade gliomas using deep learning

PurposeVolumetric assessments, such as extent of resection (EOR) or residual tumor volume, are essential criterions in glioma resection surgery. Our goal is to develop and validate segmentation machine learning models for pre- and postoperative magnetic resonance imaging scans, allowing us to assess the percentagewise tumor reduction after intracranial surgery for gliomas.MethodsFor the development of the preoperative segmentation model (U-Net), MRI scans of 1053 patients from the Multimodal Brain Tumor Segmentation Challenge (BraTS) 2021 as well as from patients who underwent surgery at the University Hospital in Zurich were used. Subsequently, the model was evaluated on a holdout set containing 285 images from the same sources. The postoperative model was developed using 72 scans and validated on 45 scans obtained from the BraTS 2015 and Zurich dataset. Performance is evaluated using Dice Similarity score, Jaccard coefficient and Hausdorff 95%.ResultsWe were able to achieve an overall mean Dice Similarity Score of 0.59 and 0.29 on the pre- and postoperative holdout sets, respectively. Our algorithm managed to determine correct EOR in 44.1%.ConclusionAlthough our models are not suitable for clinical use at this point, the possible applications are vast, going from automated lesion detection to disease progression evaluation. Precise determination of EOR is a challenging task, but we managed to show that deep learning can provide fast and objective estimates.

Clinical Decision Support Framework for Segmentation and Classification of Brain Tumor MRIs Using a U-Net and DCNN Cascaded Learning Algorithm.

Brain tumors (BTs) are an uncommon but fatal kind of cancer. Therefore, the development of computer-aided diagnosis (CAD) systems for classifying brain tumors in magnetic resonance imaging (MRI) has been the subject of many research papers so far. However, research in this sector is still in its early stage. The ultimate goal of this research is to develop a lightweight effective implementation of the U-Net deep network for use in performing exact real-time segmentation. Moreover, a simplified deep convolutional neural network (DCNN) architecture for the BT classification is presented for automatic feature extraction and classification of the segmented regions of interest (ROIs). Five convolutional layers, rectified linear unit, normalization, and max-pooling layers make up the DCNN's proposed simplified architecture. The introduced method was verified on multimodal brain tumor segmentation (BRATS 2015) datasets. Our experimental results on BRATS 2015 acquired Dice similarity coefficient (DSC) scores, sensitivity, and classification accuracy of 88.8%, 89.4%, and 88.6% for high-grade gliomas. When it comes to segmenting BRATS 2015 BT images, the performance of our proposed CAD framework is on par with existing state-of-the-art methods. However, the accuracy achieved in this study for the classification of BT images has improved upon the accuracy reported in prior studies. Image classification accuracy for BRATS 2015 BT has been improved from 88% to 88.6%.

The multimodal MRI brain tumor segmentation based on AD-Net

Multimodal glioma images provide different features of tumor boundaries based on magnetic resonance imaging (MRI), where multimodal features are often challenging to extract for deep learning segmentation methods. Disturbance between features of different modes is an important factor restricting multimodal learning. To efficiently extract multimodal features, we propose an automatic weighted dilated convolutional network (AD-Net) to learn multimodal brain tumor features through channel feature separation learning. Specifically, the auto-weight dilated convolutional unit (AD unit) utilizes dual-scale convolutional feature maps to acquire channel separation features. We employ two learnable parameters to fuse dual-scale convolutional feature maps in encoding layers, and the two learnable parameters are automatically adjusted with the back propagation of the gradient. We adopt the Jensen–Shannon divergence to constrain the distribution of its feature map, which in turn regularizes the weights of the entire down-sampling. In addition, we use the training technique of deep supervision to achieve fast fitting. Our proposed method got dice scores of 0.90, 0.80, and 0.76 for the whole tumor (WT), the tumor core (TC), and the enhancing tumor (ET) on the BraTS20 dataset. The experimental results showed good performance with the AD-Net network.

Self-Supervised Multi-Modal Hybrid Fusion Network for Brain Tumor Segmentation

Accurate medical image segmentation of brain tumors is necessary for the diagnosing, monitoring, and treating disease. In recent years, with the gradual emergence of multi-sequence magnetic resonance imaging (MRI), multi-modal MRI diagnosis has played an increasingly important role in the early diagnosis of brain tumors by providing complementary information for a given lesion. Different MRI modalities vary significantly in context, as well as in coarse and fine information. As the manual identification of brain tumors is very complicated, it usually requires the lengthy consultation of multiple experts. The automatic segmentation of brain tumors from MRI images can thus greatly reduce the workload of doctors and buy more time for treating patients. In this paper, we propose a multi-modal brain tumor segmentation framework that adopts the hybrid fusion of modality-specific features using a self-supervised learning strategy. The algorithm is based on a fully convolutional neural network. Firstly, we propose a multi-input architecture that learns independent features from multi-modal data, and can be adapted to different numbers of multi-modal inputs. Compared with single-modal multi-channel networks, our model provides a better feature extractor for segmentation tasks, which learns cross-modal information from multi-modal data. Secondly, we propose a new feature fusion scheme, named hybrid attentional fusion. This scheme enables the network to learn the hybrid representation of multiple features and capture the correlation information between them through an attention mechanism. Unlike popular methods, such as feature map concatenation, this scheme focuses on the complementarity between multi-modal data, which can significantly improve the segmentation results of specific regions. Thirdly, we propose a self-supervised learning strategy for brain tumor segmentation tasks. Our experimental results demonstrate the effectiveness of the proposed model against other state-of-the-art multi-modal medical segmentation methods.

Deep learning and machine learning‐based early survival predictions of glioblastoma patients using pre‐operative three‐dimensional brain magnetic resonance imaging modalities

AbstractTo propose and implement an automated machine learning (ML) based methodology to predict the overall survival of glioblastoma multiforme (GBM) patients. In the proposed methodology, we used deep learning (DL) based 3D U‐shaped Convolutional Neural Network inspired encoder‐decoder architecture to segment the brain tumor. Further, feature extraction was performed on these segmented and raw magnetic resonance imaging (MRI) scans using a pre‐trained 2D residual neural network. The dimension‐reduced principal components were integrated with clinical data and the handcrafted features of tumor subregions to compare the performance of regression‐based automated ML techniques. Through the proposed methodology, we achieved the mean squared error (MSE) of 87 067.328, median squared error of 30 915.66, and a SpearmanR correlation of 0.326 for survival prediction (SP) with the validation set of Multimodal Brain Tumor Segmentation 2020 dataset. These results made the MSE far better than the existing automated techniques for the same patients. Automated SP of GBM patients is a crucial topic with its relevance in clinical use. The results proved that DL‐based feature extraction using 2D pre‐trained networks is better than many heavily trained 3D and 2D prediction models from scratch. The ensembled approach has produced better results than single models. The most crucial feature affecting GBM patients' survival is the patient's age, as per the feature importance plots presented in this work. The most critical MRI modality for SP of GBM patients is the T2 fluid attenuated inversion recovery, as evident from the feature importance plots.

Particle Swarm Optimization and Two-Way Fixed-Effects Analysis of Variance for Efficient Brain Tumor Segmentation.

Simple SummarySegmentation of brain tumor images from magnetic resonance imaging (MRI) is a challenging topic in medical image analysis. The brain tumor can take many shapes, and MRI images vary considerably in intensity, making lesion detection difficult for radiologists. This paper proposes a three-step approach to solving this problem: (1) pre-processing, based on morphological operations, is applied to remove the skull bone from the image; (2) the particle swarm optimization (PSO) algorithm, with a two-way fixed-effects analysis of variance (ANOVA)-based fitness function, is used to find the optimal block containing the brain lesion; (3) the K-means clustering algorithm is adopted, to classify the detected block as tumor or non-tumor. An extensive experimental analysis, including visual and statistical evaluations, was conducted, using two MRI databases: a private database provided by the Kouba imaging center—Algiers (KICA)—and the multimodal brain tumor segmentation challenge (BraTS) 2015 database. The results show that the proposed methodology achieved impressive performance, compared to several competing approaches.Segmentation of brain tumor images, to refine the detection and understanding of abnormal masses in the brain, is an important research topic in medical imaging. This paper proposes a new segmentation method, consisting of three main steps, to detect brain lesions using magnetic resonance imaging (MRI). In the first step, the parts of the image delineating the skull bone are removed, to exclude insignificant data. In the second step, which is the main contribution of this study, the particle swarm optimization (PSO) technique is applied, to detect the block that contains the brain lesions. The fitness function, used to determine the best block among all candidate blocks, is based on a two-way fixed-effects analysis of variance (ANOVA). In the last step of the algorithm, the K-means segmentation method is used in the lesion block, to classify it as a tumor or not. A thorough evaluation of the proposed algorithm was performed, using: (1) a private MRI database provided by the Kouba imaging center—Algiers (KICA); (2) the multimodal brain tumor segmentation challenge (BraTS) 2015 database. Estimates of the selected fitness function were first compared to those based on the sum-of-absolute-differences (SAD) dissimilarity criterion, to demonstrate the efficiency and robustness of the ANOVA. The performance of the optimized brain tumor segmentation algorithm was then compared to the results of several state-of-the-art techniques. The results obtained, by using the Dice coefficient, Jaccard distance, correlation coefficient, and root mean square error (RMSE) measurements, demonstrated the superiority of the proposed optimized segmentation algorithm over equivalent techniques.

3D Multimodal Brain Tumor Segmentation and Grading Scheme based on Machine, Deep, and Transfer Learning Approaches

Glioma is one of the most common tumors of the brain. The detection and grading of glioma at an early stage is very critical for increasing the survival rate of the patients. Computer-aided detection (CADe) and computer-aided diagnosis (CADx) systems are essential and important tools that provide more accurate and systematic results to speed up the decision-making process of clinicians. In this paper, we introduce a method consisting of the variations of the machine, deep, and transfer learning approaches for the effective brain tumor (i.e., glioma) segmentation and grading on the multimodal brain tumor segmentation (BRATS) 2020 dataset. We apply popular and efficient 3D U-Net architecture for the brain tumor segmentation phase. We also utilize 23 different combinations of deep feature sets and machine learning/fine-tuned deep learning CNN models based on Xception, IncResNetv2, and EfficientNet by using 4 different feature sets and 6 learning models for the tumor grading phase. The experimental results demonstrate that the proposed method achieves 99.5% accuracy rate for slice-based tumor grading on BraTS 2020 dataset. Moreover, our method is found to have competitive performance with similar recent works.