Multimodal Image Segmentation Research Articles

Cross-dimensional transfer learning in medical image segmentation with deep learning.

Over the last decade, convolutional neural networks have emerged and advanced the state-of-the-art in various image analysis and computer vision applications. The performance of 2D image classification networks is constantly improving and being trained on databases made of millions of natural images. Conversely, in the field of medical image analysis, the progress is also remarkable but has mainly slowed down due to the relative lack of annotated data and besides, the inherent constraints related to the acquisition process. These limitations are even more pronounced given the volumetry of medical imaging data. In this paper, we introduce an efficient way to transfer the efficiency of a 2D classification network trained on natural images to 2D, 3D uni- and multi-modal medical image segmentation applications. In this direction, we designed novel architectures based on two key principles: weight transfer by embedding a 2D pre-trained encoder into a higher dimensional U-Net, and dimensional transfer by expanding a 2D segmentation network into a higher dimension one. The proposed networks were tested on benchmarks comprising different modalities: MR, CT, and ultrasound images. Our 2D network ranked first on the CAMUS challenge dedicated to echo-cardiographic data segmentation and surpassed the state-of-the-art. Regarding 2D/3D MR and CT abdominal images from the CHAOS challenge, our approach largely outperformed the other 2D-based methods described in the challenge paper on Dice, RAVD, ASSD, and MSSD scores and ranked third on the online evaluation platform. Our 3D network applied to the BraTS 2022 competition also achieved promising results, reaching an average Dice score of 91.69% (91.22%) for the whole tumor, 83.23% (84.77%) for the tumor core and 81.75% (83.88%) for enhanced tumor using the approach based on weight (dimensional) transfer. Experimental and qualitative results illustrate the effectiveness of our methods for multi-dimensional medical image segmentation.

A Hierarchical Fusion SAR Image Change-Detection Method Based on HF-CRF Model

The mainstream methods for change detection in synthetic-aperture radar (SAR) images use difference images to define the initial change regions. However, methods can suffer from semantic collapse, which makes it difficult to determine semantic information about the changes. In this paper, we proposed a hierarchical fusion SAR image change-detection model based on hierarchical fusion conditional random field (HF-CRF). This model introduces multimodal difference images and constructs the fusion energy potential function using dynamic convolutional neural networks and sliding window entropy information. By using an iterative convergence process, the proposed method was able to accurately detect the change-detection regions. We designed a dynamic region convolutional semantic segmentation network with a two-branch structure (D-DRUNet) to accomplish feature fusion and the segmentation of multimodal difference images. The proposed network adopts a dual encoder–single decoder structure where the baseline is the UNet network that utilizes dynamic convolution kernels. D-DRUNet extracts multimodal difference features and completes semantic-level fusion. The Sobel operator is introduced to strengthen the multimodal difference-image boundary information and construct the dynamic fusion pairwise potential function, based on local boundary entropy. Finally, the final change result is stabilized by iterative convergence of the CRF energy potential function. Experimental results demonstrate that the proposed method outperforms existing methods in terms of the overall number of detection errors, and reduces the occurrence of false positives.

Energy-Based MRI Semantic Augmented Segmentation for Unpaired CT Images

The multimodal segmentation of medical images is essential for clinical applications as it allows medical professionals to detect anomalies, monitor treatment effectiveness, and make informed therapeutic decisions. However, existing segmentation methods depend on paired images of modalities, which may not always be available in practical scenarios, thereby limiting their applicability. To address this challenge, current approaches aim to align modalities or generate missing modality images without a ground truth, which can introduce irrelevant texture details. In this paper, we propose the energy-basedsemantic augmented segmentation (ESAS) model, which employs the energy of latent semantic features from a supporting modality to enhance the segmentation performance on unpaired query modality data. The proposed ESAS model is a lightweight and efficient framework suitable for most unpaired multimodal image-learning tasks. We demonstrate the effectiveness of our ESAS model on the MM-WHS 2017 challenge dataset, where it significantly improved Dice accuracy for cardiac segmentation on CT volumes. Our results highlight the potential of the proposed ESAS model to enhance patient outcomes in clinical settings by providing a promising approach for unpaired multimodal medical image segmentation tasks.

A modality-collaborative convolution and transformer hybrid network for unpaired multi-modal medical image segmentation with limited annotations.

Multi-modal learning is widely adopted to learn the latent complementary information between different modalities in multi-modal medical image segmentation tasks. Nevertheless, the traditional multi-modal learning methods require spatially well-aligned and paired multi-modal images for supervised training, which cannot leverage unpaired multi-modal images with spatial misalignment and modality discrepancy. For training accurate multi-modal segmentation networks using easily accessible and low-cost unpaired multi-modal images in clinical practice, unpaired multi-modal learning has received comprehensive attention recently. Existing unpaired multi-modal learning methods usually focus on the intensity distribution gap but ignore the scale variation problem between different modalities. Besides, within existing methods, shared convolutional kernels are frequently employed to capture common patterns in all modalities, but they are typically inefficient at learning global contextual information. On the other hand, existing methods highly rely on a large number of labeled unpaired multi-modal scans for training, which ignores the practical scenario when labeled data is limited. To solve the above problems, we propose a modality-collaborative convolution and transformer hybrid network (MCTHNet) using semi-supervised learning for unpaired multi-modal segmentation with limited annotations, which not only collaboratively learns modality-specific and modality-invariant representations, but also could automatically leverage extensive unlabeled scans for improving performance. We make three main contributions to the proposed method. First, to alleviate the intensity distribution gap and scale variation problems across modalities, we develop a modality-specific scale-aware convolution (MSSC) module that can adaptively adjust the receptive field sizes and feature normalization parameters according to the input. Secondly, we propose a modality-invariant vision transformer (MIViT) module as the shared bottleneck layer for all modalities, which implicitly incorporates convolution-like local operations with the global processing of transformers for learning generalizable modality-invariant representations. Third, we design a multi-modal cross pseudo supervision (MCPS) method for semi-supervised learning, which enforces the consistency between the pseudo segmentation maps generated by two perturbed networks to acquire abundant annotation information from unlabeled unpaired multi-modal scans. Extensive experiments are performed on two unpaired CT and MR segmentation datasets, including a cardiac substructure dataset derived from the MMWHS-2017 dataset and an abdominal multi-organ dataset consisting of the BTCV and CHAOS datasets. Experiment results show that our proposed method significantly outperforms other existing state-of-the-art methods under various labeling ratios, and achieves a comparable segmentation performance close to single-modal methods with fully labeled data by only leveraging a small portion of labeled data. Specifically, when the labeling ratio is 25%, our proposed method achieves overall mean DSC values of 78.56% and 76.18% in cardiac and abdominal segmentation, respectively, which significantly improves the average DSC value of two tasks by 12.84% compared to single-modal U-Net models. Our proposed method is beneficial for reducing the annotation burden of unpaired multi-modal medical images in clinical applications.

Optimal segmentation and fusion of multi-modal brain images using clustering based deep learning algorithm

In the last several years, the world of medical technology has seen a boom in multimodal picture fusion. Information is constrained since a single medical instrument can only acquire single modal pictures. Doctors often need a large number of multimodal pictures to get the complete information necessary for disease diagnosis. The burden associated with illness diagnosis will significantly rise when multimodal pictures are employed directly, and errors and interference are likely to occur. Fusion algorithms, which have been extensively employed in the medical industry, may very effectively combine a lot of information in multimodal pictures. However, the existing method has an issue with the earlier stages of brain tumor prediction in white images and inaccuracy image results. To overcome the above-mentioned problems, in this work, Adaptive Firefly Optimization based Convolutional Neural Network (AFFOCNN) and Modified Fully Connected Layer (MFCL) scheme is proposed. This work contains main steps such as noise removal, segmentation, feature extraction, image fusion, and image classification process. Initially, noise removal is done for improving the image quality by removing the noise. Then the modality MRI images are segmented and it is used for subdividing an image into its constituent regions or object. It segments the image into black and white images. After that, feature extraction is applied through the AFFOCNN algorithm which extracts the more informative image features. Image fusion of multi-modal images derived the lower-level, middle-level, and higher-level image contents. It can be viewed in multiple directions and fused in all directions. Finally, image classification is performed by using a Modified Fully Connected Layer (MFCL) which improves the training and testing features efficiently. It was determined from the results that the suggested combination of AFFOCN and MFCL algorithm performs improved than the current algorithms with the increased accuracy, precision, recall, and mean square error (MSE), as well as execution time with the values of 99.00%, 98.00%, 96.00%, 12.00% and 2.40 seconds respectively.

Automated multi-modal Transformer network (AMTNet) for 3D medical images segmentation

Objective. Over the past years, convolutional neural networks based methods have dominated the field of medical image segmentation. But the main drawback of these methods is that they have difficulty representing long-range dependencies. Recently, the Transformer has demonstrated super performance in computer vision and has also been successfully applied to medical image segmentation because of the self-attention mechanism and long-range dependencies encoding on images. To the best of our knowledge, only a few works focus on cross-modalities of image segmentation using the Transformer. Hence, the main objective of this study was to design, propose and validate a deep learning method to extend the application of Transformer to multi-modality medical image segmentation. Approach. This paper proposes a novel automated multi-modal Transformer network termed AMTNet for 3D medical image segmentation. Especially, the network is a well-modeled U-shaped network architecture where many effective and significant changes have been made in the feature encoding, fusion, and decoding parts. The encoding part comprises 3D embedding, 3D multi-modal Transformer, and 3D Co-learn down-sampling blocks. Symmetrically, the 3D Transformer block, upsampling block, and 3D-expanding blocks are included in the decoding part. In addition, a Transformer-based adaptive channel interleaved Transformer feature fusion module is designed to fully fuse features of different modalities. Main results. We provide a comprehensive experimental analysis of the Prostate and BraTS2021 datasets. The results show that our method achieves an average DSC of 0.907 and 0.851 (0.734 for ET, 0.895 for TC, and 0.924 for WT) on these two datasets, respectively. These values show that AMTNet yielded significant improvements over the state-of-the-art segmentation networks. Significance. The proposed 3D segmentation network exploits complementary features of different modalities during the feature extraction process at multiple scales to increase the 3D feature representations and improve the segmentation efficiency. This powerful network enriches the research of the Transformer to multi-modal medical image segmentation.

Towards bridging the distribution gap: Instance to Prototype Earth Mover's Distance for distribution alignment.

Despite remarkable success of deep learning, distribution divergence remains a challenge that hinders the performance of many tasks in medical image analysis. Large distribution gap may deteriorate the knowledge transfer across different domains or feature subspaces. To achieve better distribution alignment, we propose a novel module named Instance to Prototype Earth Mover's Distance (I2PEMD), where shared class-specific prototypes are progressively learned to narrow the distribution gap across different domains or feature subspaces, and Earth Mover's Distance (EMD) is calculated to take into consideration the cross-class relationships during embedding alignment. We validate the effectiveness of the proposed I2PEMD on two different tasks: multi-modal medical image segmentation and semi-supervised classification. Specifically, in multi-modal medical image segmentation, I2PEMD is explicitly utilized as a distribution alignment regularization term to supervise the model training process, while in semi-supervised classification, I2PEMD works as an alignment measure to sort and cherry-pick the unlabeled data for more accurate and robust pseudo-labeling. Results from comprehensive experiments demonstrate the efficacy of the present method.

Learning Deformable Image Registration From Optimization: Perspective, Modules, Bilevel Training and Beyond.

Conventional deformable registration methods aim at solving an optimization model carefully designed on image pairs and their computational costs are exceptionally high. In contrast, recent deep learning-based approaches can provide fast deformation estimation. These heuristic network architectures are fully data-driven and thus lack explicit geometric constraints which are indispensable to generate plausible deformations, e.g., topology-preserving. Moreover, these learning-based approaches typically pose hyper-parameter learning as a black-box problem and require considerable computational and human effort to perform many training runs. To tackle the aforementioned problems, we propose a new learning-based framework to optimize a diffeomorphic model via multi-scale propagation. Specifically, we introduce a generic optimization model to formulate diffeomorphic registration and develop a series of learnable architectures to obtain propagative updating in the coarse-to-fine feature space. Further, we propose a new bilevel self-tuned training strategy, allowing efficient search of task-specific hyper-parameters. This training strategy increases the flexibility to various types of data while reduces computational and human burdens. We conduct two groups of image registration experiments on 3D volume datasets including image-to-atlas registration on brain MRI data and image-to-image registration on liver CT data. Extensive results demonstrate the state-of-the-art performance of the proposed method with diffeomorphic guarantee and extreme efficiency. We also apply our framework to challenging multi-modal image registration, and investigate how our registration to support the down-streaming tasks for medical image analysis including multi-modal fusion and image segmentation.

E-DU: Deep neural network for multimodal medical image segmentation based on semantic gap compensation

BackgroundU-Net includes encoder, decoder and skip connection structures. It has become the benchmark network in medical image segmentation. However, the direct fusion of low-level and high-level convolution features with semantic gaps by traditional skip connections may lead to problems such as fuzzy generated feature maps and target region segmentation errors. ObjectiveWe use spatial enhancement filtering technology to compensate for the semantic gap and propose an enhanced dense U-Net (E-DU), aiming to apply it to multimodal medical image segmentation to improve the segmentation performance and efficiency. MethodsBefore combining encoder and decoder features, we replace the traditional skip connection with a multiscale denoise enhancement (MDE) module. The encoder features need to be deeply convolved by the spatial enhancement filter and then combined with the decoder features. We propose a simple and efficient deep full convolution network structure E-DU, which can not only fuse semantically various features but also denoise and enhance the feature map. ResultsWe performed experiments on medical image segmentation datasets with seven image modalities and combined MDE with various baseline networks to perform ablation studies. E-DU achieved the best segmentation results on evaluation indicators such as DSC on the U-Net family, with DSC values of 97.78, 97.64, 95.31, 94.42, 94.93, 98.85, and 98.38 (%), respectively. The addition of the MDE module to the attention mechanism network improves segmentation performance and efficiency, reflecting its generalization performance. In comparison to advanced methods, our method is also competitive. ConclusionOur proposed MDE module has a good segmentation effect and operating efficiency, and it can be easily extended to multiple modal medical segmentation datasets. Our idea and method can achieve clinical multimodal medical image segmentation and make full use of image information to provide clinical decision support. It has great application value and promotion prospects.

MLRD-Net: 3D multiscale local cross-channel residual denoising network for MRI-based brain tumor segmentation

The precise segmentation of multimodal MRI images is the primary stage of tumor diagnosis and treatment. Current segmentation strategies often underutilize multiscale features, which can easily lead to loss of contextual information, reduction of low-level features and noise interference. To overcome these issues, a 3D multiscale local cross-channel residual denoising network (MLRD-Net) for an MRI-based brain tumor segmentation algorithm is proposed in this paper. Specifically, we employ encoder-decoder structure to connect local and global features, and enhance the receptive field of the network. Random slice operation has been conducted to enhance robustness. Then, residual blocks with pre-activation operation are developed in down-sampling stage, which effectively improves signal propagation along the network and alleviates network overfitting. Finally, the local cross-channel denoising mechanism is established to eliminate unimportant features without dimensionality reduction. Our proposal was evaluated in Brain Tumor Segmentation 2020 dataset (BraTS 2020), obtaining significantly improved results with mean Dice Similarity Coefficient metric of 0.91, 0.79, and 0.73 for the complete, tumor core, and enhancing tumor regions respectively. Besides, we conduct further practice on BraTS 2019, with the mean Dice Similarity Coefficient metric of 0.89, 0.80, and 0.75. Massive experiments demonstrate that our method is powerful and reliable. It increases little model complexity while achieving very competitive performance.

LMISA: A lightweight multi-modality image segmentation network via domain adaptation using gradient magnitude and shape constraint.

In medical image segmentation, supervised machine learning models trained using one image modality (e.g. computed tomography (CT)) are often prone to failure when applied to another image modality (e.g. magnetic resonance imaging (MRI)) even for the same organ. This is due to the significant intensity variations of different image modalities. In this paper, we propose a novel end-to-end deep neural network to achieve multi-modality image segmentation, where image labels of only one modality (source domain) are available for model training and the image labels for the other modality (target domain) are not available. In our method, a multi-resolution locally normalized gradient magnitude approach is firstly applied to images of both domains for minimizing the intensity discrepancy. Subsequently, a dual task encoder-decoder network including image segmentation and reconstruction is utilized to effectively adapt a segmentation network to the unlabeled target domain. Additionally, a shape constraint is imposed by leveraging adversarial learning. Finally, images from the target domain are segmented, as the network learns a consistent latent feature representation with shape awareness from both domains. We implement both 2D and 3D versions of our method, in which we evaluate CT and MRI images for kidney and cardiac tissue segmentation. For kidney, a public CT dataset (KiTS19, MICCAI 2019) and a local MRI dataset were utilized. The cardiac dataset was from the Multi-Modality Whole Heart Segmentation (MMWHS) challenge 2017. Experimental results reveal that our proposed method achieves significantly higher performance with a much lower model complexity in comparison with other state-of-the-art methods. More importantly, our method is also capable of producing superior segmentation results than other methods for images of an unseen target domain without model retraining. The code is available at GitHub (https://github.com/MinaJf/LMISA) to encourage method comparison and further research.

EGMM: An evidential version of the Gaussian mixture model for clustering

The Gaussian mixture model (GMM) provides a simple yet principled framework for clustering, with properties suitable for statistical inference. In this paper, we propose a new model-based clustering algorithm, called EGMM (evidential GMM), in the theoretical framework of belief functions to better characterize cluster-membership uncertainty. With a mass function representing the cluster membership of each object, the evidential Gaussian mixture distribution composed of the components over the powerset of the desired clusters is proposed to model the entire dataset. The parameters in EGMM are estimated by a specially designed Expectation–Maximization (EM) algorithm. A validity index allowing automatic determination of the proper number of clusters is also provided. The proposed EGMM is as simple as the classical GMM, but can generate a more informative evidential partition for the considered dataset. The synthetic and real dataset experiments show that the proposed EGMM performs better than other representative clustering algorithms. Besides, its superiority is also demonstrated by an application to multi-modal brain image segmentation.

DeepImageTranslator V2: Analysis of Multimodal Medical Images using Semantic Segmentation Maps Generated through Deep Learning

Introduction: Analysis of multimodal medical images often requires the selection of one or many anatomical regions of interest (ROIs) for extraction of useful statistics. This task can prove laborious when a manual approach is used. We have previously developed a user-friendly software tool for image-to-image translation using deep learning. Therefore, we present herein an update to the DeepImageTranslator V2 software with the addition of a tool for multimodal medical image segmentation analysis (hereby referred to as the MMMISA). Methods: The MMMISA was implemented using the Tkinter library; backend computations were implemented using the Pydicom, Numpy, and OpenCV libraries. We tested our software using 4188 slices from whole-body axial 2-deoxy-2-[18F]-fluoroglucose-position emission tomography/ computed tomography scans ([¹⁸F]-FDG-PET/CT) of 10 patients from the American College of Radiology Imaging Network-Head and Neck Squamous Cell Carcinoma (ACRIN-HNSCC) database. Using the deep learning software DeepImageTranslator, a model was trained with 36 randomly selected CT slices and manually labelled semantic segmentation maps. Utilizing the trained model, all the CT scans of the 10 HNSCC patients were segmented with high accuracy. Segmentation maps generated using the deep convolutional network were then used to measure organ specific [¹⁸F]-FDG uptake. We also compared measurements performed using the MMMISA and those made with manually selected ROIs. Results: The MMMISA is a tool that allows user to select ROIs based on deep learning-generated segmentation maps and to compute accurate statistics for these ROIs based on coregistered multimodal images. We found that organ-specific [¹⁸F]-FDG uptake measured using multiple manually selected ROIs is concordant with whole-tissue measurements made with segmentation maps using the MMMISA tool. Doi: 10.28991/HIJ-2022-03-03-07 Full Text: PDF

3D Automatic Segmentation of Brain Tumor Based on Deep Neural Network and Multimodal MRI Images.

Brain tumor segmentation is an important content in medical image processing, and it is also a very common research in medicine. Due to the development of modern technology, it is very valuable to use deep learning (DL) and multimodal MRI images to study brain tumor segmentation. In order to solve the problems of low efficiency and low accuracy of brain tumor segmentation, this paper proposes DL to conduct research on multimodal MRI image segmentation, aiming to make accurate diagnosis and treatment for doctors. In addition, this paper constructs an automatic diagnosis system for brain tumors, uses GLCM and discrete wavelet transform (DWT) to extract features from MRI images, and then uses a convolutional neural network (CNN) for final diagnosis; finally, through four. The comparison of the results between the two algorithms proves that the CNN algorithm has the better processing power and higher efficiency.

A bidirectional multilayer contrastive adaptation network with anatomical structure preservation for unpaired cross-modality medical image segmentation

Multi-modal medical image segmentation has achieved great success through supervised deep learning networks. However, because of domain shift and limited annotation information, unpaired cross-modality segmentation tasks are still challenging. The unsupervised domain adaptation (UDA) methods can alleviate the segmentation degradation of cross-modality segmentation by knowledge transfer between different domains, but current methods still suffer from the problems of model collapse, adversarial training instability, and mismatch of anatomical structures. To tackle these issues, we propose a bidirectional multilayer contrastive adaptation network (BMCAN) for unpaired cross-modality segmentation. The shared encoder is first adopted for learning modality-invariant encoding representations in image synthesis and segmentation simultaneously. Secondly, to retain the anatomical structure consistency in cross-modality image synthesis, we present a structure-constrained cross-modality image translation approach for image alignment. Thirdly, we construct a bidirectional multilayer contrastive learning approach to preserve the anatomical structures and enhance encoding representations, which utilizes two groups of domain-specific multilayer perceptron (MLP) networks to learn modality-specific features. Finally, a semantic information adversarial learning approach is designed to learn structural similarities of semantic outputs for output space alignment. Our proposed method was tested on three different cross-modality segmentation tasks: brain tissue, brain tumor, and cardiac substructure segmentation. Compared with other UDA methods, experimental results show that our proposed BMCAN achieves state-of-the-art segmentation performance on the above three tasks, and it has fewer training components and better feature representations for overcoming overfitting and domain shift problems. Our proposed method can efficiently reduce the annotation burden of radiologists in cross-modality image analysis.

Prior wavelet knowledge for multi-modal medical image segmentation using a lightweight neural network with attention guided features

Medical image segmentation plays a crucial role in diagnosing and staging diseases. It facilitates image analysis and quantification in multiple applications, but building the right appropriate solutions is essential and highly reliant on the features of different datasets and computational resources. Most existing approaches provide segmentation for a specific anatomical region of interest and are limited to multiple imaging modalities in a clinical setting due to their generalizability with high computational requirements. To mitigate these issues, we propose a robust and lightweight deep learning real-time segmentation network for multi-modality medical images called MISegNet. We incorporate discrete wavelet transform (DWT) of the input to extract salient features in the frequency domain. This mechanism allows the neurons’ receptive field to enlarge within the network. We propose a self-attention-based global context-aware (SGCA) module with varying dilation rates to enlarge the field of view and designate the importance of each scale that enhances the network’s ability to discriminate features. We build a residual shuffle attention (RSA) mechanism to improve the feature representation of the proposed model and formulate a new boundary-aware loss function called Farid End Point Error (FEPE) that correctly segments regions with ambiguous boundaries for edge detection. We confirm the versatility of the proposed model by performing experiments against eleven state-of-the-art segmentation methods on four datasets of different organs, including two publicly available datasets (i.e., ISBI2017, and COVID-19 CT) and two private datasets (i.e., ovary and liver ultrasound images). Experimental results prove that the MISegNet with 1.5M parameters, outperforms the state-of-the-art methods by 1.5%–7% (i.e., dice coefficient score) with a corresponding 23× decrease in the number of parameters and multiply-accumulate operations respectively compared to U-Net.

Dual attention-guided and learnable spatial transformation data augmentation multi-modal unsupervised medical image segmentation

Unsupervised domain adaptation multi-modal medical image segmentation method is used for joint training to realize the segmentation of different modal medical images at the same time. Since the domain shift of different modal images and the limited labeled medical images, the accuracy of these methods needs to be further improved. In this work, we present a novel unsupervised domain adaptation method, named as Dual Attention-guided and Learnable spatial transformation data Augmentation multi-modal unsupervised medical image segmentation (DALA). Firstly, this paper mainly introduces the position and channel Dual Attention Mechanism (Dual Attent-M) into the low-level encoder to improve the feature extraction ability of the network and enhance the domain adaptation training of the network. Secondly, a learnable Spatial Transformation data Augmentation method (Spatial Tran-Aug) is further proposed to learn the spatial mapping relationship between the source image and the target image to synthesize high-quality data for training. Experiments on the Multi-Modality Whole Heart Segmentation (MMWHS) dataset show that compared with the multi-modal segmentation methods such as PnP-AdaNet, SynSeg-Net, AdaOutput, CyCADA, Prior SIF and SIFA, the proposed method DALA can achieve better segmentation results, and the average DICE predicted by CT and MR is increased to 78.2% and 67.9%, the mean ASSD decreased to 4.4 and 4.7.

Automatic Segmentation of MRI of Brain Tumor Using Deep Convolutional Network.

Computer-aided diagnosis and treatment of multimodal magnetic resonance imaging (MRI) brain tumor image segmentation has always been a hot and significant topic in the field of medical image processing. Multimodal MRI brain tumor image segmentation utilizes the characteristics of each modal in the MRI image to segment the entire tumor and tumor core area and enhanced them from normal brain tissues. However, the grayscale similarity between brain tissues in various MRI images is very immense making it difficult to deal with the segmentation of multimodal MRI brain tumor images through traditional algorithms. Therefore, we employ the deep learning method as a tool to make full use of the complementary feature information between the multimodalities and instigate the following research: (i) build a network model suitable for brain tumor segmentation tasks based on the fully convolutional neural network framework and (ii) adopting an end-to-end training method, using two-dimensional slices of MRI images as network input data. The problem of unbalanced categories in various brain tumor image data is overcome by introducing the Dice loss function into the network to calculate the network training loss; at the same time, parallel Dice loss is proposed to further improve the substructure segmentation effect. We proposed a cascaded network model based on a fully convolutional neural network to improve the tumor core area and enhance the segmentation accuracy of the tumor area and achieve good prediction results for the substructure segmentation on the BraTS 2017 data set.

Brain Tumor Image Segmentation Based on Grouped Convolution

The automatic segmentation of MRI multi-modal images of brain tumors is one of the important research contents of disease detection and analysis. Due to the heterogeneity of tumors, it is difficult to achieve efficient and accurate automatic segmentation of brain tumors. Traditional segmentation methods based on machine learning cannot handle complex scenes such as complex edges and overlapping categories. In clinical assisted diagnosis, it is of great significance to apply deep learning to two-dimensional natural image segmentation and three-dimensional medical image segmentation. In this paper, we propose a three-dimensional network model to achieve precise segmentation of brain tumors. The model adopts an encoder-decoder structure and replaces ordinary convolution with grouped convolution to reduce network parameters and improve network performance. The model improves the problem of information exchange between different groups through channel mixing. Experiments conducted on the BraTS (Brain Tumor Segmentation) 2018 challenge dataset prove that our network greatly reduces the computational cost while ensuring segmentation accuracy.

Medical Image Segmentation Algorithm for Three-Dimensional Multimodal Using Deep Reinforcement Learning and Big Data Analytics.

To avoid the problems of relative overlap and low signal-to-noise ratio (SNR) of segmented three-dimensional (3D) multimodal medical images, which limit the effect of medical image diagnosis, a 3D multimodal medical image segmentation algorithm using reinforcement learning and big data analytics is proposed. Bayesian maximum a posteriori estimation method and improved wavelet threshold function are used to design wavelet shrinkage algorithm to remove high-frequency signal component noise in wavelet domain. The low-frequency signal component is processed by bilateral filtering and the inverse wavelet transform is used to denoise the 3D multimodal medical image. An end-to-end DRD U-Net model based on deep reinforcement learning is constructed. The feature extraction capacity of denoised image segmentation is increased by changing the convolution layer in the traditional reinforcement learning model to the residual module and introducing the multiscale context feature extraction module. The 3D multimodal medical image segmentation is done using the reward and punishment mechanism in the deep learning reinforcement algorithm. In order to verify the effectiveness of 3D multimodal medical image segmentation algorithm, the LIDC-IDRI data set, the SCR data set, and the DeepLesion data set are selected as the experimental data set of this article. The results demonstrate that the algorithm's segmentation effect is effective. When the number of iterations is increased to 250, the structural similarity reaches 98%, the SNR is always maintained between 55 and 60 dB, the training loss is modest, relative overlap and accuracy all exceed 95%, and the overall segmentation performance is superior. Readers will understand how deep reinforcement learning and big data analytics test the effectiveness of 3D multimodal medical image segmentation algorithm.