Framework For Image Segmentation Research Articles

MACCoM: A multiple attention and convolutional cross-mixer framework for detailed 2D biomedical image segmentation

The UNet architecture, which is widely used for biomedical image segmentation, has limitations like blurred feature maps and over- or under-segmented regions. To overcome these limitations, we propose a novel network architecture called MACCoM (Multiple Attention and Convolutional Cross-Mixer) – an end-to-end depthwise encoder-decoder fully convolutional network designed for binary and multi-class biomedical image segmentation built upon deeperUNet. We proposed a multi-scope attention module (MSAM) that allows the model to attend to diverse scale features, preserving fine details and high-level semantic information thus useful at the encoder-decoder connection. As the depth increases, our proposed spatial multi-head attention (SMA) is added to facilitate inter-layer communication and information exchange, enabling the network to effectively capture long-range dependencies and global context. MACCoM is also equipped with a convolutional cross-mixer we proposed to enhance the feature extraction capability of the model. By incorporating these modules, we effectively combine semantically similar features and reduce artifacts during the early stages of training. Experimental results on 4 biomedical datasets crafted from 3 datasets of varying modalities consistently demonstrate that MACCoM outperforms or matches state-of-the-art baselines in the segmentation tasks. With Breast Ultrasound Image (BUSI), MACCoM recorded 99.06 % Jaccard, 77.58 % Dice, and 93.92 % Accuracy, while recording 99.50 %, 98.44 %, and 99.29 % respectively for Jaccard, Dice, and Accuracy on the Chest X-ray (CXR) images used. The Jaccard, Dice, and Accuracy for the High-Resolution Fundus (HRF) images are 95.77 %, 74.35 %, and 95.95 % respectively. The findings here highlight MACCoM's effectiveness in improving segmentation performance and its valuable potential in image analysis.

HiDiff: Hybrid Diffusion Framework for Medical Image Segmentation.

Medical image segmentation has been significantly advanced with the rapid development of deep learning (DL) techniques. Existing DL-based segmentation models are typically discriminative; i.e., they aim to learn a mapping from the input image to segmentation masks. However, these discriminative methods neglect the underlying data distribution and intrinsic class characteristics, suffering from unstable feature space. In this work, we propose to complement discriminative segmentation methods with the knowledge of underlying data distribution from generative models. To that end, we propose a novel hybrid diffusion framework for medical image segmentation, termed HiDiff, which can synergize the strengths of existing discriminative segmentation models and new generative diffusion models. HiDiff comprises two key components: discriminative segmentor and diffusion refiner. First, we utilize any conventional trained segmentation models as discriminative segmentor, which can provide a segmentation mask prior for diffusion refiner. Second, we propose a novel binary Bernoulli diffusion model (BBDM) as the diffusion refiner, which can effectively, efficiently, and interactively refine the segmentation mask by modeling the underlying data distribution. Third, we train the segmentor and BBDM in an alternate-collaborative manner to mutually boost each other. Extensive experimental results on abdomen organ, brain tumor, polyps, and retinal vessels segmentation datasets, covering four widely-used modalities, demonstrate the superior performance of HiDiff over existing medical segmentation algorithms, including the state-of-the-art transformer- and diffusion-based ones. In addition, HiDiff excels at segmenting small objects and generalizing to new datasets. Source codes are made available at https://github.com/takimailto/HiDiff.

HiDiff: Hybrid Diffusion Framework for Medical Image Segmentation.

Medical image segmentation has been significantly advanced with the rapid development of deep learning (DL) techniques. Existing DL-based segmentation models are typically discriminative; i.e., they aim to learn a mapping from the input image to segmentation masks. However, these discriminative methods neglect the underlying data distribution and intrinsic class characteristics, suffering from unstable feature space. In this work, we propose to complement discriminative segmentation methods with the knowledge of underlying data distribution from generative models. To that end, we propose a novel hybrid diffusion framework for medical image segmentation, termed HiDiff, which can synergize the strengths of existing discriminative segmentation models and new generative diffusion models. HiDiff comprises two key components: discriminative segmentor and diffusion refiner. First, we utilize any conventional trained segmentation models as discriminative segmentor, which can provide a segmentation mask prior for diffusion refiner. Second, we propose a novel binary Bernoulli diffusion model (BBDM) as the diffusion refiner, which can effectively, efficiently, and interactively refine the segmentation mask by modeling the underlying data distribution. Third, we train the segmentor and BBDM in an alternate-collaborative manner to mutually boost each other. Extensive experimental results on abdomen organ, brain tumor, polyps, and retinal vessels segmentation datasets, covering four widely-used modalities, demonstrate the superior performance of HiDiff over existing medical segmentation algorithms, including the state-of-the-art transformer- and diffusion-based ones. In addition, HiDiff excels at segmenting small objects and generalizing to new datasets. Source codes are made available at https://github.com/takimailto/HiDiff.

A multi-level closing based segmentation framework for dermatoscopic images using ensemble deep network

A multi-level closing based segmentation framework for dermatoscopic images using ensemble deep network

DIAS: A dataset and benchmark for intracranial artery segmentation in DSA sequences

The automated segmentation of Intracranial Arteries (IA) in Digital Subtraction Angiography (DSA) plays a crucial role in the quantification of vascular morphology, significantly contributing to computer-assisted stroke research and clinical practice. Current research primarily focuses on the segmentation of single-frame DSA using proprietary datasets. However, these methods face challenges due to the inherent limitation of single-frame DSA, which only partially displays vascular contrast, thereby hindering accurate vascular structure representation. In this work, we introduce DIAS, a dataset specifically developed for IA segmentation in DSA sequences. We establish a comprehensive benchmark for evaluating DIAS, covering full, weak, and semi-supervised segmentation methods. Specifically, we propose the vessel sequence segmentation network, in which the sequence feature extraction module effectively captures spatiotemporal representations of intravascular contrast, achieving intracranial artery segmentation in 2D+Time DSA sequences. For weakly-supervised IA segmentation, we propose a novel scribble learning-based image segmentation framework, which, under the guidance of scribble labels, employs cross pseudo-supervision and consistency regularization to improve the performance of the segmentation network. Furthermore, we introduce the random patch-based self-training framework, aimed at alleviating the performance constraints encountered in IA segmentation due to the limited availability of annotated DSA data. Our extensive experiments on the DIAS dataset demonstrate the effectiveness of these methods as potential baselines for future research and clinical applications. The dataset and code are publicly available at https://doi.org/10.5281/zenodo.11401368 and https://github.com/lseventeen/DIAS.

BiU-net: A dual-branch structure based on two-stage fusion strategy for biomedical image segmentation

Background and objectiveComputer-based biomedical image segmentation plays a crucial role in planning of assisted diagnostics and therapy. However, due to the variable size and irregular shape of the segmentation target, it is still a challenge to construct an effective medical image segmentation structure. Recently, hybrid architectures based on convolutional neural networks (CNNs) and transformers were proposed. However, most current backbones directly replace one or all convolutional layers with transformer blocks, regardless of the semantic gap between features. Thus, how to sufficiently and effectively eliminate the semantic gap as well as combine the global and local information is a critical challenge. MethodsTo address the challenge, we propose a novel structure, called BiU-Net, which integrates CNNs and transformers with a two-stage fusion strategy. In the first fusion stage, called Single-Scale Fusion (SSF) stage, the encoding layers of the CNNs and transformers are coupled, with both having the same feature map size. The SSF stage aims to reconstruct local features based on CNNs and long-range information based on transformers in each encoding block. In the second stage, Multi-Scale Fusion (MSF), BiU-Net interacts with multi-scale features from various encoding layers to eliminate the semantic gap between deep and shallow layers. Furthermore, a Context-Aware Block (CAB) is embedded in the bottleneck to reinforce multi-scale features in the decoder. ResultsExperiments on four public datasets were conducted. On the BUSI dataset, our BiU-Net achieved 85.50 % on Dice coefficient (Dice), 76.73 % on intersection over union (IoU), and 97.23 % on accuracy (ACC). Compared to the state-of-the-art method, BiU-Net improves Dice by 1.17 %. For the Monuseg dataset, the proposed method attained the highest scores, reaching 80.27 % and 67.22 % for Dice and IoU. The BiU-Net achieves 95.33 % and 81.22 % Dice on the PH2 and DRIVE datasets. ConclusionsThe results of our experiments showed that BiU-Net transcends existing state-of-the-art methods on four publicly available biomedical datasets. Due to the powerful multi-scale feature extraction ability, our proposed BiU-Net is a versatile medical image segmentation framework for various types of medical images. The source code is released on (https://github.com/ZYLandy/BiU-Net).

DA-TransUNet: integrating spatial and channel dual attention with transformer U-net for medical image segmentation.

Accurate medical image segmentation is critical for disease quantification and treatment evaluation. While traditional U-Net architectures and their transformer-integrated variants excel in automated segmentation tasks. Existing models also struggle with parameter efficiency and computational complexity, often due to the extensive use of Transformers. However, they lack the ability to harness the image's intrinsic position and channel features. Research employing Dual Attention mechanisms of position and channel have not been specifically optimized for the high-detail demands of medical images. To address these issues, this study proposes a novel deep medical image segmentation framework, called DA-TransUNet, aiming to integrate the Transformer and dual attention block (DA-Block) into the traditional U-shaped architecture. Also, DA-TransUNet tailored for the high-detail requirements of medical images, optimizes the intermittent channels of Dual Attention (DA) and employs DA in each skip-connection to effectively filter out irrelevant information. This integration significantly enhances the model's capability to extract features, thereby improving the performance of medical image segmentation. DA-TransUNet is validated in medical image segmentation tasks, consistently outperforming state-of-the-art techniques across 5 datasets. In summary, DA-TransUNet has made significant strides in medical image segmentation, offering new insights into existing techniques. It strengthens model performance from the perspective of image features, thereby advancing the development of high-precision automated medical image diagnosis. The codes and parameters of our model will be publicly available at https://github.com/SUN-1024/DA-TransUnet.

Model-data-driven adversarial active learning for brain tumor segmentation

Active learning (AL) attempts to select informative samples in a dataset to minimize the number of required labels while maximizing the performance of the model. Current AL in segmentation tasks is limited to the expansion of popular classification-based methods including entropy, MC-dropout, etc. Meanwhile, most applications in the medical field are simply migrations that fail to consider the nature of medical images, such as high class imbalance, high domain difference, and data scarcity. In this study, we address these challenges and propose a novel AL framework for medical image segmentation task. Our approach introduces a pseudo-label-based filter addressing excessive blank patches in medical abnormalities segmentation tasks, e.g., lesions, and tumors, used before the AL selection. This filter helps reduce resource usage and allows the model to focus on selecting more informative samples. For the sample selection, we propose a novel query strategy that combines both model impact and data stability by employing adversarial attack. Furthermore, we harness the adversarial samples generated during the query process to enhance the robustness of the model. The experimental results verify our framework’s effectiveness over various state-of-the-art methods. Our proposed method only needs less than 14% annotated patches in 3D brain MRI multiple sclerosis (MS) segmentation tasks and 20% for Low-Grade Glioma (LGG) tumor segmentation to achieve competitive results with full supervision. These promising outcomes not only improve performance but alleviate the time burden associated with expert annotation, thereby facilitating further advancements in the field of medical image segmentation. Our code is available at https://github.com/HelenMa9998/adversarial_active_learning.

Chaotic opposition Golden Sinus Algorithm for global optimization problems

Optimization techniques are required to find the best solutions to challenging problems in many engineering disciplines. Metaheuristic algorithms that effectively increase search performance using various evolutionary strategies have become increasingly popular in recent years. The Golden Sinus Algorithm (GoldSa) is a population-based optimization algorithm that uses the sine function and the golden ratio. In this study, a chaotically enhanced opposition-based Golden Sinus Algorithm (Co-GoldSa) has been proposed to improve the efficiency of the exploitation and exploration ability of the GoldSa method. While designing this approach, it is first necessary to analyze the chaotic behavior of the GoldSa parameters. For this purpose, three chaotic GoldSa methods have been developed using eight chaotic maps to determine the effect of the chaotic maps on the parameters with different behaviors. Secondly, the opposition-based learning strategy is adjusted to the cGoldSa to enhance the searching ability. To investigate the proficiency of the proposed Co-GoldSa method, it has been examined with well-known and newly introduced metaheuristic approaches on benchmark functions and classical engineering design problems. Besides, an efficient framework of the multilevel thresholding image segmentation has been presented based on the Co-GoldSa method since the efficient processing of pathological images is quite important in medical diagnostics. The experimental outcomes reveal the superiority of the proposed method in solving global optimization problems, image segmentation, and engineering problems. Thus, the outcomes of the benchmark functions, image segmentation, and classical engineering problems support that the proposed Co-GoldSa approach can be considered a promising method for resolving challenging optimization problems.

Differentiable double clustering with edge-aware superpixel fitting for unsupervised image segmentation

Unsupervised image segmentation is an essential topic in the field of computer vision, which broke the limitation of training data and expanded application scenarios. Off-the-shelf clustering methods simply rely on semantic concepts and incomplete boundary cues, resulting in incorrect segmentation in object boundaries. Therefore, this paper proposes an unsupervised image segmentation framework combining differentiable double clustering (DDC) and edge-aware superpixel (EA), which outperform prior work on the accuracy of the prior art. First, a multi-layer feature extraction network is introduced to combine low-level and high-level features for clustering. Then, the DDC module is designed to obtain initial labels of pixels from both local and global perspectives to improve the clustering accuracy. Pixel-wise feature similarity in different classes is pushed away, and one in the same class is brought closer. Finally, we use EA to provide well-fitting boundary cues for DDC label fusion to reduce incorrect segmentation. Extensive experiments on the benchmark datasets PASCAL VOC2012 [34] and BSD500 demonstrate that the proposed method provides competitive segmentation results.

Self-training with dual uncertainty for semi-supervised MRI image segmentation

The goal of semi-supervised MRI image segmentation is to train high-accuracy segmentation models using a small amount of labeled data and a large amount of unlabeled data, thereby alleviating the burden of manual annotation by physicians. Self-training is a traditional semi-supervised learning method aimed at acquiring additional image features by generating pseudo-labels for unlabeled data. However, this method may lead to an unstable training process due to differences between samples. Additionally, the predictions of model may contain erroneous noise, leading to low-confidence pseudo-labels. To address these issues, we proposed a dual uncertainty-guided self-training framework (DUST) for semi-supervised MRI image segmentation. Specifically, the two uncertainties consist of sample-level uncertainty and pixel-level uncertainty. The sample-level uncertainty is intended to achieve a stable and smooth training process. The pixel-level uncertainty is intended to rectify pseudo-labels. We conducted a series of experiments on two public MRI image datasets, ACDC2017 and Prostate2018. Compared to the baseline, we improved the Dice scores by 5.0% and 4.0% over the two datasets, respectively. Furthermore, the experimental results indicate that the proposed method has certain advantages compared to the comparative methods. This validates the feasibility and applicability of our method.

Multi-Source Domain Adaptation for Medical Image Segmentation.

Unsupervised domain adaptation(UDA) aims to mitigate the performance drop of models tested on the target domain, due to the domain shift from the target to sources. Most UDA segmentation methods focus on the scenario of solely single source domain. However, in practical situations data with gold standard could be available from multiple sources (domains), and the multi-source training data could provide more information for knowledge transfer. How to utilize them to achieve better domain adaptation yet remains to be further explored. This work investigates multi-source UDA and proposes a new framework for medical image segmentation. Firstly, we employ a multi-level adversarial learning scheme to adapt features at different levels between each of the source domains and the target, to improve the segmentation performance. Then, we propose a multi-model consistency loss to transfer the learned multi-source knowledge to the target domain simultaneously. Finally, we validated the proposed framework on two applications, i.e., multi-modality cardiac segmentation and cross-modality liver segmentation. The results showed our method delivered promising performance and compared favorably to state-of-the-art approaches.

VisionDeep-AI: Deep learning-based retinal blood vessels segmentation and multi-class classification framework for eye diagnosis

Eye problems can lead to vision loss and have a significant impact on daily life, underscoring the critical importance of early diagnosis and treatment to prevent further damage and complications. A comprehensive segmentation-classification framework – VisionDeep-AI is developed in the proposed work for retinal vessel segmentation and multi-class classification on given fundus images. A weighted bi-directional feature pyramid network and U-Net backbone architecture-based customized deep-learning model is built to segment blood vessels which enhances feature extraction and multi-scale feature fusion. The architectural design comprises an end-to-end encoder-decoder network featuring six-depth layers with varying resolutions, enabling the extraction of high-level descriptors and lower-level, fine-grained characteristics. Furthermore, a multi-modal deep feature fusion architecture is developed for the multi-class classification of fundus images in four different categories by integrating features from segmented vessel images and raw fundus images, thereby accommodating more diversified information. A thorough analysis of VisionDeep-AI has been done on a comprehensive colour fundus image dataset. To ensure reliable results, data augmentation was done to prevent the over-fitting of the model and to enhance its generalization capability. The segmentation model performed exceptionally well and achieved a high accuracy of 97.73% and a dice coefficient of 89.90% in blood vessel segmentation from fundus images. Furthermore, multi-modal deep feature-fused classification architecture achieved a test accuracy of 81.50%, with a specificity of 93.83%. These findings showcase the robustness, generalizability and efficiency of the proposed VisionDeep-AI framework for fundus image segmentation and retinal disease classification, contributing towards advanced medical technologies and diagnosis precision.

CFR-ICL: Cascade-Forward Refinement with Iterative Click Loss for Interactive Image Segmentation

The click-based interactive segmentation aims to extract the object of interest from an image with the guidance of user clicks. Recent work has achieved great overall performance by employing feedback from the output. However, in most state-of-the-art approaches, 1) the inference stage involves inflexible heuristic rules and requires a separate refinement model, and 2) the number of user clicks and model performance cannot be balanced. To address the challenges, we propose a click-based and mask-guided interactive image segmentation framework containing three novel components: Cascade-Forward Refinement (CFR), Iterative Click Loss (ICL), and SUEM image augmentation. The CFR offers a unified inference framework to generate segmentation results in a coarse-to-fine manner. The proposed ICL allows model training to improve segmentation and reduce user interactions simultaneously. The proposed SUEM augmentation is a comprehensive way to create large and diverse training sets for interactive image segmentation. Extensive experiments demonstrate the state-of-the-art performance of the proposed approach on five public datasets. Remarkably, our model reduces by 33.2%, and 15.5% the number of clicks required to surpass an IoU of 0.95 in the previous state-of-the-art approach on the Berkeley and DAVIS sets, respectively.

FishSegSSL: A Semi-Supervised Semantic Segmentation Framework for Fish-Eye Images.

The application of large field-of-view (FoV) cameras equipped with fish-eye lenses brings notable advantages to various real-world computer vision applications, including autonomous driving. While deep learning has proven successful in conventional computer vision applications using regular perspective images, its potential in fish-eye camera contexts remains largely unexplored due to limited datasets for fully supervised learning. Semi-supervised learning comes as a potential solution to manage this challenge. In this study, we explore and benchmark two popular semi-supervised methods from the perspective image domain for fish-eye image segmentation. We further introduce FishSegSSL, a novel fish-eye image segmentation framework featuring three semi-supervised components: pseudo-label filtering, dynamic confidence thresholding, and robust strong augmentation. Evaluation on the WoodScape dataset, collected from vehicle-mounted fish-eye cameras, demonstrates that our proposed method enhances the model's performance by up to 10.49% over fully supervised methods using the same amount of labeled data. Our method also improves the existing image segmentation methods by 2.34%. To the best of our knowledge, this is the first work on semi-supervised semantic segmentation on fish-eye images. Additionally, we conduct a comprehensive ablation study and sensitivity analysis to showcase the efficacy of each proposed method in this research.

An improved RIME optimization algorithm for lung cancer image segmentation

Lung cancer is a prevalent form of cancer worldwide, necessitating early and accurate diagnosis for successful treatment. Within medical imaging processing, image segmentation plays a vital role in medical diagnosis. This study applies swarm intelligence algorithms to segment lung cancer pathological images at three levels. The original algorithm incorporates the Whales' search prey mechanism and a random mutation strategy, resulting in an improved version named WDRIME, which aims to enhance convergence speed and avoid local optima (LO). Additionally, the study introduces a multilevel image segmentation method for lung cancer based on the improved algorithm. WDRIME's performance is showcased by comparing it to the state-of-the-art algorithms in IEEE CEC2014. To design a framework for lung cancer image segmentation, this paper combines the WDRIME algorithm with the multilevel segmentation method. Evaluation of the segmentation results employs metrics such as PSNR, SSIM, and FSIM. Overall, the analysis confirms that the proposed algorithm supersedes others regarding convergence speed and accuracy. This model signifies a high-quality segmentation method and offers practical support for in-depth exploration of lung cancer pathological images.

Light-M: An efficient lightweight medical image segmentation framework for resource-constrained IoMT

The Internet of Medical Things (IoMT) is being incorporated into current healthcare systems. This technology intends to connect patients, IoMT devices, and hospitals over mobile networks, allowing for more secure, quick, and convenient health monitoring and intelligent healthcare services. However, existing intelligent healthcare applications typically rely on large-scale AI models, and standard IoMT devices have significant resource constraints. To alleviate this paradox, in this paper, we propose a Knowledge Distillation (KD)-based IoMT end-edge-cloud orchestrated architecture for medical image segmentation tasks, called Light-M, aiming to deploy a lightweight medical model in resource-constrained IoMT devices. Specifically, Light-M trains a large teacher model in the cloud server and employs computation in local nodes through imitation of the performance of the teacher model using knowledge distillation. Light-M contains two KD strategies: (1) active exploration and passive transfer (AEPT) and (2) self-attention-based inter-class feature variation (AIFV) distillation for the medical image segmentation task. The AEPT encourages the student model to learn undiscovered knowledge/features of the teacher model without additional feature layers, aiming to explore new features and outperform the teacher. To improve the distinguishability of the student for different classes, the student learns the self-attention-based feature variation (AIFV) between classes. Since the proposed AEPT and AIFV only appear in the training process, our framework does not involve any additional computation burden for a student model during the segmentation task deployment. Extensive experiments on cardiac images and public real-scene datasets demonstrate that our approach improves student model learning representations and outperforms state-of-the-art methods by combining two knowledge distillation strategies. Moreover, when deployed on the IoT device, the distilled student model takes only 29.6 ms for one sample at the inference step.

An Implicit-Explicit Prototypical Alignment Framework for Semi-Supervised Medical Image Segmentation.

Semi-supervised learning methods have been explored to mitigate the scarcity of pixel-level annotation in medical image segmentation tasks. Consistency learning, serving as a mainstream method in semi-supervised training, suffers from low efficiency and poor stability due to inaccurate supervision and insufficient feature representation. Prototypical learning is one potential and plausible way to handle this problem due to the nature of feature aggregation in prototype calculation. However, the previous works have not fully studied how to enhance the supervision quality and feature representation using prototypical learning under the semi-supervised condition. To address this issue, we propose an implicit-explicit alignment (IEPAlign) framework to foster semi-supervised consistency training. In specific, we develop an implicit prototype alignment method based on dynamic multiple prototypes on-the-fly. And then, we design a multiple prediction voting strategy for reliable unlabeled mask generation and prototype calculation to improve the supervision quality. Afterward, to boost the intra-class consistency and inter-class separability of pixel-wise features in semi-supervised segmentation, we construct a region-aware hierarchical prototype alignment, which transmits information from labeled to unlabeled and from certain regions to uncertain regions. We evaluate IEPAlign on three medical image segmentation tasks. The extensive experimental results demonstrate that the proposed method outperforms other popular semi-supervised segmentation methods and achieves comparable performance with fully-supervised training methods.

Self-supervised spatial–temporal transformer fusion based federated framework for 4D cardiovascular image segmentation

Availability of high-quality large annotated data is indeed a significant challenge in healthcare. In addition, privacy concerns and data-sharing restrictions often hinder access to large and diverse medical image datasets. To reduce the requirement for annotated training data, self-supervised pre-training strategies on nonannotated data have been extensively used, whereas collaborative algorithm training without the need to exchange the underlying data. In this paper, we introduce a novel federated learning-based self-supervised spatial–temporal transformer’s fusion (SSFL) for cardiovascular image segmentation. The integration of spatial–temporal swin transformer is used to extract the features from 3D SAX multiple phases (full cycle of cardiac heart). An efficient self-supervised contrastive framework consisting of a spatial–temporal transformer network with 25 encoders is used to model the temporal features. The spatial and temporal features are fused and forwarded to the decoder for cardiac heart segmentation using cine MRI images. To further improve segmentation, we use an attention-based unpaired GAN model to map or transfer the style from ACDC to M&Ms and use synthetically generated volumes in the proposed self-supervised approach. Experiments with three different cardiovascular image segmentation tasks, such as segmentation of the right ventricle, left ventricle, and myocardium, showed significant improvement compared to the state-of-the-art segmentation framework.

Neighbouring-slice Guided Multi-View Framework for brain image segmentation

Automated segmentation of brain images is a critical task in neuroscience for brain registration, atlas generation, etc. Deep learning techniques have been widely investigated for segmenting brain images, where most existing methods only consider each brain slice independently in a single view, limiting their ability to explore the correlation among adjacent slices and spatial brain structures. This paper proposes a Neighbouring-slice Guided Multi-view Framework for automated segmentation of brain images. To fully utilize the information between neighbouring slices, we design a dual-decoder network to segment targets (e.g., regions/tumors) in brain images and the edge of each brain targets simultaneously, by calculating the difference between adjacent slices. Considering the fact that some brain images cannot be fully recognized in a single view, we integrate the neighbouring-slice strategy in a multi-view segmentation framework to fully explore spatial structures in 3D brains. The proposed framework is validated on the automated segmentation of diverse brain tumors and brain regions including CTX (cerebellar cortex), CP (caudoputamen), HPF (hippocampal formation), BS (brain stem), CB (cerebellum), and CBX (cerebellar cortex), in LSFM (Light-sheet Fluorescence Microscopy) and MRI (Magnetic Resonance Imaging) modalities, demonstrating superior performance in comparison with state-of-the-arts. The codes are released at: https://github.com/NeuronXJTU/NGMV.