Tumor Segmentation Research Articles

Two-Stage Training Framework Using Multicontrast MRI Radiomics for IDH Mutation Status Prediction in Glioma.

Purpose To develop a radiomics framework for preoperative MRI-based prediction of isocitrate dehydrogenase (IDH) mutation status, a crucial glioma prognostic indicator. Materials and Methods Radiomics features (shape, first-order statistics, and texture) were extracted from the whole tumor or the combination of nonenhancing, necrosis, and edema regions. Segmentation masks were obtained via the federated tumor segmentation tool or the original data source. Boruta, a wrapper-based feature selection algorithm, identified relevant features. Addressing the imbalance between mutated and wild-type cases, multiple prediction models were trained on balanced data subsets using random forest or XGBoost and assembled to build the final classifier. The framework was evaluated using retrospective MRI scans from three public datasets (The Cancer Imaging Archive [TCIA, 227 patients], the University of California San Francisco Preoperative Diffuse Glioma MRI dataset [UCSF, 495 patients], and the Erasmus Glioma Database [EGD, 456 patients]) and internal datasets collected from the University of Texas Southwestern Medical Center (UTSW, 356 patients), New York University (NYU, 136 patients), and University of Wisconsin-Madison (UWM, 174 patients). TCIA and UTSW served as separate training sets, while the remaining data constituted the test set (1617 or 1488 testing cases, respectively). Results The best performing models trained on the TCIA dataset achieved area under the receiver operating characteristic curve (AUC) values of 0.89 for UTSW, 0.86 for NYU, 0.93 for UWM, 0.94 for UCSF, and 0.88 for EGD test sets. The best performing models trained on the UTSW dataset achieved slightly higher AUCs: 0.92 for TCIA, 0.88 for NYU, 0.96 for UWM, 0.93 for UCSF, and 0.90 for EGD. Conclusion This MRI radiomics-based framework shows promise for accurate preoperative prediction of IDH mutation status in patients with glioma. Keywords: Glioma, Isocitrate Dehydrogenase Mutation, IDH Mutation, Radiomics, MRI Supplemental material is available for this article. Published under a CC BY 4.0 license. See also commentary by Moassefi and Erickson in this issue.

Revolutionizing Brain Tumor Analysis: A Fusion of ChatGPT and Multi-Modal CNN for Unprecedented Precision

In this study, we introduce an innovative approach to significantly enhance the precision and interpretability of brain tumor detection and segmentation. Our method ingeniously integrates the cutting-edge capabilities of the ChatGPT chatbot interface with a state-of-the-art multi-modal convolutional neural network (CNN). Tested rigorously on the BraTS dataset, our method showcases unprecedented performance, outperforming existing techniques in terms of both accuracy and efficiency, with an impressive Dice score of 0.89 for tumor segmentation. By seamlessly integrating ChatGPT, our model unveils deep-seated insights into the intricate decision-making processes, providing researchers and physicians with invaluable understanding and confidence in the results. This groundbreaking fusion holds immense promise, poised to revolutionize the landscape of medical imaging, with far-reaching implications for clinical practice and research. Our study exemplifies the transformative potential achieved through the synergistic combination of multi-modal CNNs and natural language processing, paving the way for remarkable advancements in brain tumor detection and segmentation.

HCA-DAN: hierarchical class-aware domain adaptive network for gastric tumor segmentation in 3D CT images

BackgroundAccurate segmentation of gastric tumors from CT scans provides useful image information for guiding the diagnosis and treatment of gastric cancer. However, automated gastric tumor segmentation from 3D CT images faces several challenges. The large variation of anisotropic spatial resolution limits the ability of 3D convolutional neural networks (CNNs) to learn features from different views. The background texture of gastric tumor is complex, and its size, shape and intensity distribution are highly variable, which makes it more difficult for deep learning methods to capture the boundary. In particular, while multi-center datasets increase sample size and representation ability, they suffer from inter-center heterogeneity.MethodsIn this study, we propose a new cross-center 3D tumor segmentation method named Hierarchical Class-Aware Domain Adaptive Network (HCA-DAN), which includes a new 3D neural network that efficiently bridges an Anisotropic neural network and a Transformer (AsTr) for extracting multi-scale context features from the CT images with anisotropic resolution, and a hierarchical class-aware domain alignment (HCADA) module for adaptively aligning multi-scale context features across two domains by integrating a class attention map with class-specific information. We evaluate the proposed method on an in-house CT image dataset collected from four medical centers and validate its segmentation performance in both in-center and cross-center test scenarios.ResultsOur baseline segmentation network (i.e., AsTr) achieves best results compared to other 3D segmentation models, with a mean dice similarity coefficient (DSC) of 59.26%, 55.97%, 48.83% and 67.28% in four in-center test tasks, and with a DSC of 56.42%, 55.94%, 46.54% and 60.62% in four cross-center test tasks. In addition, the proposed cross-center segmentation network (i.e., HCA-DAN) obtains excellent results compared to other unsupervised domain adaptation methods, with a DSC of 58.36%, 56.72%, 49.25%, and 62.20% in four cross-center test tasks.ConclusionsComprehensive experimental results demonstrate that the proposed method outperforms compared methods on this multi-center database and is promising for routine clinical workflows.

Towards consistency in pediatric brain tumor measurements: Challenges, solutions, and the role of artificial intelligence-based segmentation.

MR imaging is central to the assessment of tumor burden and changes over time in neuro-oncology. Several response assessment guidelines have been set forth by the Response Assessment in Pediatric Neuro-Oncology (RAPNO) working groups in different tumor histologies; however, the visual delineation of tumor components using MRIs is not always straightforward, and complexities not currently addressed by these criteria can introduce inter- and intra-observer variability in manual assessments. Differentiation of non-enhancing tumors from peritumoral edema, mild enhancement from absence of enhancement, and various cystic components can be challenging; particularly given a lack of sufficient and uniform imaging protocols in clinical practice. Automated tumor segmentation with artificial intelligence (AI) may be able to provide more objective delineations, but rely on accurate and consistent training data created manually (ground truth). Herein, this paper reviews existing challenges and potential solutions to identifying and defining subregions of pediatric brain tumors (PBTs) that are not explicitly addressed by current guidelines. The goal is to assert the importance of defining and adopting criteria for addressing these challenges, as it will be critical to achieving standardized tumor measurements and reproducible response assessment in PBTs, ultimately leading to more precise outcome metrics and accurate comparisons among clinical studies.

Sailfish optimizer based CLAHE with U-NET for MRI brain tumour segmentation

Semantic brain tumour segmentation is a significant contribution to medical image processing. This research is helpful for doctors making diagnoses and determining the severity of lesions. In recent years, convolutional neural networks have sometimes been known as self-remarkable representations in computer vision tasks. In three-dimensional medical image processing, CNN with architecture and skip connections has become increasingly popular. However, they are inadequate at both acquiring nearby and far-off semantic knowledge. However, the transformer's use of a vision. This accomplishment was made possible by the global information modelling capabilities of the transformer. When tryingto perform challenging prediction tasks, such as segmenting 3D medical pictures, it is vital to consider local and global features. In this study, we propose a hypothesis that predicts pre-processing steps that enhance the image quality and significantly impact accuracy with any statistical image classification method. We used a better picture enhancement methodology to back up our idea. This method has three unique stages: a median filter to eliminate noise, a histogram equalisation technique to boost contrast, and a grayscale-to-RGB conversion. The clipping limit of contrast-limited Adaptive Histogram Equalisation (CLAHE)is calculated by the Sailfish Optimization Algorithm (SOA) to achieve optimal picture enhancement. This study classifies semantically segmenting 3D brain tumours as a sequence-to-sequence prediction problem. U-net is proposed in the article as a novel method for segmenting 3D medical images; it incorporates a structure. A new technique for segmenting 3D medical images, U-net, is emerging. We trained and assessed our method on the BRATS 2019 dataset, which was found to outperform state-of-the-art algorithms and be competitive under similar settings.

Deep Learning Analysis of 3D Volume of Brain Anomaly Using MRI

In this study, we present a comprehensive investigation into the efficacy of two state-of-the-art deep learning architectures, 3D UNet and VNet, for brain tumor detection and segmentation in volumetric medical images.

Comprehensive Review on MRI-Based Brain Tumor Segmentation: A Comparative Study from 2017 Onwards

Comprehensive Review on MRI-Based Brain Tumor Segmentation: A Comparative Study from 2017 Onwards

TACA-RNet: Tri-Axis Based Context-Aware Reverse Network for Multimodal Brain Tumor Segmentation

Brain tumor segmentation using Magnetic Resonance Imaging (MRI) is vital for clinical decision making. Traditional deep learning-based studies using convolutional neural networks have predominantly processed MRI data as two-dimensional slices, leading to the loss of contextual information. While three-dimensional (3D) convolutional layers represent an advancement, they have not fully exploited pathological information according to the three-axis nature of 3D MRI data—axial, coronal, and sagittal. Recognizing these limitations, we introduce a Tri-Axis based Context-Aware Reverse Network (TACA-RNet). This innovative approach leverages the unique 3D spatial orientations of MRI, learning crucial information on brain anatomy and pathology. We incorporated three specialized modules: a Tri-Axis Channel Reduction module for optimizing feature dimensions, a MultiScale Contextual Fusion module for aggregating multi-scale features and enhancing spatial discernment, and a 3D Axis Reverse Attention module for the precise delineation of tumor boundaries. The TACA-RNet leverages three specialized modules to enhance the understanding of tumor characteristics and spatial relationships within MRI data by fully utilizing its tri-axial structure. Validated on the Brain Tumor Segmentation Challenge 2018 and 2020 datasets, the TACA-RNet demonstrated superior performances over contemporary methodologies. This underscores the critical role of leveraging the three-axis structure of MRI to enhance segmentation accuracy.

PSO-PSP-Net + InceptionV3: An optimized hyper-parameter tuned Computer-Aided Diagnostic model for liver tumor detection using CT scan slices

An automated diagnostic system leads to a supreme requirement in medical image analysis, greatly impacting the death rate due to the high spreading rate of liver tumors. However, the traditional histopathological diagnostic processes cause more disruption in the medical detection framework because of the procedure of manual estimation of CT scans and medical labor-intensive, which lead to the time consumption process. It is a highly challenging task for the segmentation of liver tumor because of the irregular appearance, distinct densities, patchy edges, and tumor sizes. To overcome this issue, an automated computer-aided diagnosis framework has been utilized to diagnose the liver cancer quickly and accurately with the less workload of expert radiologists.Presently, Deep learning-based techniques have been applied to identify the tumor regions and assist radiologists in the initial diagnosis and scrutinizing the severity level. The proposed work presents a modified pyramid scene parsing network (PSP-Net) model based on InceptionV3 deep network architecture with a particle swarm optimization algorithm, which increases the efficiency and effectiveness of the liver diagnostic model. Three publicly accessible benchmark datasets have been engaged with the pre-processing task accomplished by image normalization and data augmentation techniques. Moreover, a hyper-parameter tuned improved PSO-PSP-Net algorithm has analyzed the optimal set of gathered features. In the experimental evaluation, the proposed model outperformed than grey wolf optimization (GWO-PSP-Net), PSO-U-Net, GWO-U-Net, opposition based spotted hyena optimization (O-SHO-CRNN), and grey wolf-class topper optimization (GW-CTO) + U-Net by 8.42 %, 4.9 %, 9.54 %, 5.12 %, and 1.75 %, respectively in terms of accuracy for LiTS dataset. The developed PSO-PSP-Net model attained better accuracy than GWO-PSP-Net, PSO-U-Net, GWO-U-Net, and Residual Atrous U-Net (RA-U-Net) for the segmentation of Type-I and Type-II liver tumors by 6.88 %, 8.44 %, 13.75 %, 5.99 % and 6.12 %, 9.74 %, 17.3 %, and 5.11 %, respectively for 3DIRCAD-b1 dataset. The obtained accuracy of the developed model is 14.09 %, 7.54 %, 16.36 %, 2.64 %, and 14.9 %, 8.86 %, 17.76 %, and 4.87 % maximized than GWO-PSP-Net, PSO-U-Net, GWO-U-Net, and RA-U-Net, respectively, for the segmentation of the infected arteries and bones in the 3DIRCAD-b1 dataset. For the CHAOS dataset, the improved PSO-PSP-Net model outperformed GWO-PSP-Net, PSO-U-Net, and GWO-U-Net, respectively, in terms of overall accuracy by 7.44 %, 8.85 %, and 17.2 %, respectively. The investigational analysis validates the proposed PSO-PSP-Net model's effectiveness and performance in obtaining high segmentation accuracy in comparison with other state-of-the-art approaches.

Brain Tumor Prediction and Segmentation with Morphological Region-based Active Contour Model and Refinement using Boltzmann Monte Carlo Method in MRI Images

Objectives: The primary goal of the research work is to accurately detect the precise location of the brain tumor in the radiological Magnetic Resonance Imaging (MRI) images of human brain using segmentation method. Methods: In this research work, we introduce mainly the Morphological Region-based Active Contour model and Boltzmann Monte Carlo method (MACB model), involving a comprehensive three-step methodology for the segmentation of the brain, MRI images in order to detect brain tumor. The initial step involves pre-processing which includes Gaussian filtering for noise reduction and Contrast Limited Adaptive Histogram Equalization (CLAHE) technique to enhance image features. In the second step, we identify tumor-related clusters using morphological operations and delineate the tumor regions using Active Contour (Snake) model to get a segmented image. In the final step, the Boltzmann Monte Carlo method is used to refine the edges of the segmented image. To evaluate the effectiveness of this approach, the 2D brain tumor datasets, available in the public domain, are used. The first dataset is taken from Kaggle website and has 3064 MRI human brain images and its respective ground truth images which is used for segmentation. The second dataset is used for visualization of segmented tumor, available in the same Kaggle website. Findings: The Performance metrics for finding similarity between the segmented images generated using the proposed MACB model and the ground truth images, available in the first dataset, exhibit higher values. That is, the proposed method has achieved higher values of Dice Similarity Coefficient (DSC): 93.26%, Jaccard Co-efficient: 86.44%, Sensitivity: 97.27%, Specificity: 99.43% and Pixel accuracy: 98.95%. Novelty: In this research work, MACB model is proposed for the detection, segmentation, and refinement process of brain tumor by incorporating Boltzmann Monte Carlo method with Morphological Region-Based Active Contour model. This novel approach has resulted in enhanced precision and efficiency in the brain tumor segmentation process. Keywords: Brain Tumor Segmentation, Morphological Operation, Active Contour, Boltzmann Monte Carlo Method, Magnetic Resonance Imaging

Imaging-Based Deep Learning for Predicting Desmoid Tumor Progression.

Desmoid tumors (DTs) are non-metastasizing and locally aggressive soft-tissue mesenchymal neoplasms. Those that become enlarged often become locally invasive and cause significant morbidity. DTs have a varied pattern of clinical presentation, with up to 50-60% not growing after diagnosis and 20-30% shrinking or even disappearing after initial progression. Enlarging tumors are considered unstable and progressive. The management of symptomatic and enlarging DTs is challenging, and primarily consists of chemotherapy. Despite wide surgical resection, DTs carry a rate of local recurrence as high as 50%. There is a consensus that contrast-enhanced magnetic resonance imaging (MRI) or, alternatively, computerized tomography (CT) is the preferred modality for monitoring DTs. Each uses Response Evaluation Criteria in Solid Tumors version 1.1 (RECIST 1.1), which measures the largest diameter on axial, sagittal, or coronal series. This approach, however, reportedly lacks accuracy in detecting response to therapy and fails to detect tumor progression, thus calling for more sophisticated methods. The objective of this study was to detect unique features identified by deep learning that correlate with the future clinical course of the disease. Between 2006 and 2019, 51 patients (mean age 41.22 ± 15.5 years) who had a tissue diagnosis of DT were included in this retrospective single-center study. Each had undergone at least three MRI examinations (including a pretreatment baseline study), and each was followed by orthopedic oncology specialists for a median of 38.83 months (IQR 44.38). Tumor segmentations were performed on a T2 fat-suppressed treatment-naive MRI sequence, after which the segmented lesion was extracted to a three-dimensional file together with its DICOM file and run through deep learning software. The results of the algorithm were then compared to clinical data collected from the patients' medical files. There were 28 males (13 stable) and 23 females (15 stable) whose ages ranged from 19.07 to 83.33 years. The model was able to independently predict clinical progression as measured from the baseline MRI with an overall accuracy of 93% (93 ± 0.04) and ROC of 0.89 ± 0.08. Artificial intelligence may contribute to risk stratification and clinical decision-making in patients with DT by predicting which patients are likely to progress.

Automatic segmentation of femoral tumors by nnU-net

BackgroundMetastatic femoral tumors may lead to pathological fractures during daily activities. A CT-based finite element analysis of a patient's femurs was shown to assist orthopedic surgeons in making informed decisions about the risk of fracture and the need for a prophylactic fixation. Improving the accuracy of such analyses ruqires an automatic and accurate segmentation of the tumors and their automatic inclusion in the finite element model. We present herein a deep learning algorithm (nnU-Net) to automatically segment lytic tumors within the femur. MethodA dataset consisting of fifty CT scans of patients with manually annotated femoral tumors was created. Forty of them, chosen randomly, were used for training the nnU-Net, while the remaining ten CT scans were used for testing. The deep learning model's performance was compared to two experienced radiologists. FindingsThe proposed algorithm outperformed the current state-of-the-art solutions, achieving dice similarity scores of 0.67 and 0.68 on the test data when compared to two experienced radiologists, while the dice similarity score for inter-individual variability between the radiologists was 0.73. InterpretationThe automatic algorithm may segment lytic femoral tumors in CT scans as accurately as experienced radiologists with similar dice similarity scores. The influence of the realistic tumors inclusion in an autonomous finite element algorithm is presented in (Rachmil et al., "The Influence of Femoral Lytic Tumors Segmentation on Autonomous Finite Element Analyses", Clinical Biomechanics, 112, paper 106192, (2024)).

Efficient Brain Tumor Segmentation with Lightweight Separable Spatial Convolutional Network

Accurate and automated segmentation of lesions in brain MRI scans is crucial in diagnostics and treatment planning. Despite the significant achievements of existing approaches, they often require substantial computational resources and fail to fully exploit the synergy between low-level and high-level features. To address these challenges, we introduce the Separable Spatial Convolutional Network (SSCN), an innovative model that refines the U-Net architecture to achieve efficient brain tumor segmentation with minimal computational cost. SSCN integrates the PocketNet paradigm and replaces standard convolutions with depthwise separable convolutions, resulting in a significant reduction in parameters and computational load. Additionally, our feature complementary module enhances the interaction between features across the encoder-decoder structure, facilitating the integration of multi-scale features while maintaining low computational demands. The model also incorporates a separable spatial attention mechanism, enhancing its capability to discern spatial details. Empirical validations on standard datasets demonstrate the effectiveness of our proposed model, especially in segmenting small and medium-sized tumors, with only 0.27M parameters and 3.68 GFlops. Our code is available at https://github.com/zzpr/SSCN .

SARFNet: Selective Layer and Axial Receptive Field Network for Multimodal Brain Tumor Segmentation

Efficient magnetic resonance imaging (MRI) segmentation, which is helpful for treatment planning, is essential for identifying brain tumors from detailed images. In recent years, various convolutional neural network (CNN) structures have been introduced for brain tumor segmentation tasks and have performed well. However, the downsampling blocks of most existing methods are typically used only for processing the variation in image sizes and lack sufficient capacity for further extraction features. We, therefore, propose SARFNet, a method based on UNet architecture, which consists of the proposed SLiRF module and advanced AAM module. The SLiRF downsampling module can extract feature information and prevent the loss of important information while reducing the image size. The AAM block, incorporated into the bottleneck layer, captures more contextual information. The Channel Attention Module (CAM) is introduced into skip connections to enhance the connections between channel features to improve accuracy and produce better feature expression. Ultimately, deep supervision is utilized in the decoder layer to avoid vanishing gradients and generate better feature representations. Many experiments were performed to validate the effectiveness of our model on the BraTS2018 dataset. SARFNet achieved Dice coefficient scores of 90.40, 85.54, and 82.15 for the whole tumor (WT), tumor core (TC), and enhancing tumor (ET), respectively. The results show that the proposed model achieves state-of-the-art performance compared with twelve or more benchmarks.

Leveraging SO(3)-steerable convolutions for pose-robust semantic segmentation in 3D medical data

Convolutional neural networks (CNNs) allow for parameter sharing and translational equivariance by using convolutional kernels in their linear layers. By restricting these kernels to be SO(3)-steerable, CNNs can further improve parameter sharing and equivariance. These equivariant convolutional layers have several advantages over standard convolutional layers, including increased robustness to unseen poses, smaller network size, and improved sample efficiency. Despite this, most segmentation networks used in medical image analysis continue to rely on standard convolutional kernels. In this paper, we present a new family of segmentation networks that use equivariant voxel convolutions based on spherical harmonics. These SE(3)-equivariant volumetric segmentation networks, which are robust to data poses not seen during training, do not require rotation-based data augmentation during training. In addition, we demonstrate improved segmentation performance in MRI brain tumor and healthy brain structure segmentation tasks, with enhanced robustness to reduced amounts of training data and improved parameter efficiency. Code to reproduce our results, and to implement the equivariant segmentation networks for other tasks is available at <a href='http://github.com/SCAN-NRAD/e3nn_Unet'>http://github.com/SCAN-NRAD/e3nn_Unet</a>.

MFMSNet: A Multi-frequency and Multi-scale Interactive CNN-Transformer Hybrid Network for breast ultrasound image segmentation

Breast tumor segmentation in ultrasound images is fundamental for quantitative analysis and plays a crucial role in the diagnosis and treatment of breast cancer. Recently, existing methods have mainly focused on spatial domain implementations, with less attention to the frequency domain. In this paper, we propose a Multi-frequency and Multi-scale Interactive CNN-Transformer Hybrid Network (MFMSNet). Specifically, we utilize Octave convolutions instead of conventional convolutions to effectively separate high-frequency and low-frequency components while reducing computational complexity. Introducing the Multi-frequency Transformer block (MF-Trans) enables efficient interaction between high-frequency and low-frequency information, thereby capturing long-range dependencies. Additionally, we incorporate Multi-scale interactive fusion module (MSIF) to merge high-frequency feature maps of different sizes, enhancing the emphasis on tumor edges by integrating local contextual information. Experimental results demonstrate the superiority of our MFMSNet over seven state-of-the-art methods on two publicly available breast ultrasound datasets and one thyroid ultrasound dataset. In the evaluation of MFMSNet, tests were conducted on the BUSI, BUI, and DDTI datasets, comprising 130 images (BUSI), 47 images (BUI), and 128 images (DDTI) in the respective test sets. Employing a five-fold cross-validation approach, the obtained dice coefficients are as follows: 83.42 % (BUSI), 90.79 % (BUI), and 79.96 % (DDTI). The code is available at https://github.com/wrc990616/MFMSNet.

SSGNet: Selective Multi-Scale Receptive Field and Kernel Self-Attention Based on Group-Wise Modality for Brain Tumor Segmentation

SSGNet: Selective Multi-Scale Receptive Field and Kernel Self-Attention Based on Group-Wise Modality for Brain Tumor Segmentation

Model-data-driven adversarial active learning for brain tumor segmentation

Active learning (AL) attempts to select informative samples in a dataset to minimize the number of required labels while maximizing the performance of the model. Current AL in segmentation tasks is limited to the expansion of popular classification-based methods including entropy, MC-dropout, etc. Meanwhile, most applications in the medical field are simply migrations that fail to consider the nature of medical images, such as high class imbalance, high domain difference, and data scarcity. In this study, we address these challenges and propose a novel AL framework for medical image segmentation task. Our approach introduces a pseudo-label-based filter addressing excessive blank patches in medical abnormalities segmentation tasks, e.g., lesions, and tumors, used before the AL selection. This filter helps reduce resource usage and allows the model to focus on selecting more informative samples. For the sample selection, we propose a novel query strategy that combines both model impact and data stability by employing adversarial attack. Furthermore, we harness the adversarial samples generated during the query process to enhance the robustness of the model. The experimental results verify our framework’s effectiveness over various state-of-the-art methods. Our proposed method only needs less than 14% annotated patches in 3D brain MRI multiple sclerosis (MS) segmentation tasks and 20% for Low-Grade Glioma (LGG) tumor segmentation to achieve competitive results with full supervision. These promising outcomes not only improve performance but alleviate the time burden associated with expert annotation, thereby facilitating further advancements in the field of medical image segmentation. Our code is available at https://github.com/HelenMa9998/adversarial_active_learning.

FLAIR MRI sequence synthesis using squeeze attention generative model for reliable brain tumor segmentation

Manual segmentation of brain tumors using structural magnetic resonance imaging (MRI) is an arduous and time-consuming task. Therefore, automatic and robust segmentation will considerably influence neuro-oncological clinical trials by reducing excessive manual annotation time. Herein, we propose a deep learning model that automatically segments brain tumors even in cases of missing MRI sequences, which are common in practical clinical settings. To address this issue, we enhance a generative adversarial network (GAN) by incorporating a squeeze-and-excitation (SE) attention module into its generator and a PatchGAN into its discriminator. The SE module recalibrates channel responses by explicitly modeling interdependencies, enabling the network to focus on critical regions such as tumor areas. Our proposed generative model is optimized using a combination of adversarial, structural similarity, and mean absolute error losses to synthesize missing MRI sequences more effectively. This enhancement allows our model to synthesize the missing MRI sequence (fluid attenuated inversion recovery [FLAIR]) by leveraging information from other available sequences (T1-weighted, T2-weighted, or contrast-enhanced T1-weighted [T1ce]). For the segmentation task, we employ an optimized nnU-Net model, which is trained using existing sequences and evaluated using both available and synthesized sequences (including missing ones), mimicking real-world scenarios where often only limited MRI sequences are available or usable. Our findings reveal a notable enhancement in brain tumor segmentation, as indicated by a significant increase in overall the Dice similarity coefficient (DSC) from 0.688% (when FLAIR is missing) to 0.873% (when using synthesized FLAIR derived from T2). This improvement brings the segmentation performance closer to what was achieved when real FLAIR was available, where the DSC reaches 0.901%. Moreover, our synthesizing model was also tested on two additional datasets: the BraTS 2020 validation set and BraTS Africa 2023 training set, which produces results comparable to those of BraTS 2021, thereby proving its robustness and generalizability. In addition, the resulting tumor segmentations are subsequently employed to assess the response to treatment in cases where all sequences were available and when synthesis was employed, according to response assessment in neuro-oncology criteria.

Brain tumor segmentation with missing MRI modalities using edge aware discriminative feature fusion based transformer U-net

Brain tumor segmentation is an essential task for medical diagnosis and treatment planning. Multi-modal MRI provides complementary information that is essential for accurate segmentation of brain tumors, but missing modality images are a common problem in clinical practice. Existing segmentation methods often fail to generate accurate object boundaries and selectively fuse the tumor region, resulting in unreliable segmentation masks. In this work, we propose an Edge-aware Discriminative Feature Fusion Based Transformer U-Net (EA-DFFTU-Net) to segment the brain tumor effectively even in the absence of modalities. First, the MRI input data is pre-processed, and features are then extracted using a ResNet-50 encoder, which learns local spatial. We employ an Edge Feature Module (EFM) to acquire edge attention representations. These features are then transferred to the Discriminative Feature Fusion based Transformer U-Net (DFFTU-Net) that learns global contexts and captures multiscale features. We use a Discriminative Feature Fusion Module (DFFM) in the decoder of the DFFTU-Net to effectively fuse the multiscale features in order to obtain accurate segmentation. The performance of the proposed EA-DFFTU-Net segmentation method was determined by evaluating it and comparing the obtained results with those of existing brain tumor segmentation techniques.