Network For Image Segmentation Research Articles

SASAN: Spectrum-Axial Spatial Approach Networks for Medical Image Segmentation.

Ophthalmic diseases such as central serous chorioretinopathy (CSC) significantly impair the vision of millions of people globally. Precise segmentation of choroid and macular edema is critical for diagnosing and treating these conditions. However, existing 3D medical image segmentation methods often fall short due to the heterogeneous nature and blurry features of these conditions, compounded by medical image clarity issues and noise interference arising from equipment and environmental limitations. To address these challenges, we propose the Spectrum Analysis Synergy Axial-Spatial Network (SASAN), an approach that innovatively integrates spectrum features using the Fast Fourier Transform (FFT). SASAN incorporates two key modules: the Frequency Integrated Neural Enhancer (FINE), which mitigates noise interference, and the Axial-Spatial Elementum Multiplier (ASEM), which enhances feature extraction. Additionally, we introduce the Self-Adaptive Multi-Aspect Loss (LSM), which balances image regions, distribution, and boundaries, adaptively updating weights during training. We compiled and meticulously annotated the Choroid and Macular Edema OCT Mega Dataset (CMED-18k), currently the world's largest dataset of its kind. Comparative analysis against 13 baselines shows our method surpasses these benchmarks, achieving the highest Dice scores and lowest HD95 in the CMED and OIMHS datasets. Our code is publicly available at https://github.com/IMOP-lab/SASAN-Pytorch.

Image Semantic Segmentation Approach for Studying Human Behavior on Image Data

Image semantic segmentation is an essential technique for studying human behavior through image data. This paper proposes an image semantic segmentation method for human behavior research. Firstly, an end-to-end convolutional neural network architecture is proposed, which consists of a depth-separable jump-connected fully convolutional network and a conditional random field network; then jump-connected convolution is used to classify each pixel in the image, and an image semantic segmentation method based on convolutional neural network is proposed; and then a conditional random field network is used to improve the effect of image segmentation of human behavior and a linear modeling and nonlinear modeling method based on the semantic segmentation of conditional random field image is proposed. Finally, using the proposed image segmentation network, the input entrepreneurial image data is semantically segmented to obtain the contour features of the person; and the segmentation of the images in the medical field. The experimental results show that the image semantic segmentation method is effective. It is a new way to use image data to study human behavior and can be extended to other research areas.

PAMSNet: A medical image segmentation network based on spatial pyramid and attention mechanism

The image segmentation of diseases can help clinical diagnosis and treatment in medical image analysis. Due to the complexity of lesion features (e.g., size, location, and morphology) and the high similarity between the background and the target area in medical images, semantic features are difficult to extract completely. To tackle these problems, we propose a novel medical image segmentation network PAMSNet based on the spatial pyramid and attention mechanism. By using efficient pyramid attention and channel spatial attention modules, the proposed method fuses the extracted multi-scale spatial information with the local features extracted by the encoder to supplement the image details. In addition, the Spatial Pyramid-Coordinate Attention (SPCA) module is introduced in the bottleneck layer to obtain larger receptive field information and enhance feature extraction. We conducted qualitative and quantitative evaluations on four public datasets, including ISIC2018, Lung segmentation, Kvasir-SEG, and ISLES2022. The segmentation accuracy of DSC was 87.86%, 98.18%, 82.43%, and 87.37%, respectively. The ablation study of each part of PAMSNet proves the validity of each component, and the comparison with state-of-the-art methods on different indicators proves the predominance of the network.

SDMI-Net: Spatially Dependent Mutual Information Network for semi-supervised medical image segmentation

Semi-supervised medical image segmentation strives to polish deep models with a small amount of labeled data and a large amount of unlabeled data. The efficiency of most semi-supervised medical image segmentation methods based on voxel-level consistency learning is affected by low-confidence voxels. In addition, voxel-level consistency learning fails to consider the spatial correlation between neighboring voxels. To encourage reliable voxel-level consistent learning, we propose a dual-teacher affine consistent uncertainty estimation method to filter out some voxels with high uncertainty. Moreover, we design the spatially dependent mutual information module, which enhances the spatial dependence between neighboring voxels by maximizing the mutual information between the local voxel blocks predicted from the dual-teacher models and the student model, enabling consistent learning at the block level. On two benchmark medical image segmentation datasets, including the Left Atrial Segmentation Challenge dataset and the BraTS-2019 dataset, our method achieves state-of-the-art performance in both quantitative and qualitative aspects.

Medical image segmentation network based on multi-scale frequency domain filter

With the development of deep learning, medical image segmentation in computer-aided diagnosis has become a research hotspot. Recently, UNet and its variants have become the most powerful medical image segmentation methods. However, these methods suffer from (1) insufficient sensing field and insufficient depth; (2) computational nonlinearity and redundancy of channel features; and (3) ignoring the interrelationships among feature channels. These problems lead to poor network segmentation performance and weak generalization ability. Therefore, first of all, we propose an effective replacement scheme of UNet base block, Double residual depthwise atrous convolution (DRDAC) block, to effectively improve the deficiency of receptive field and depth. Secondly, a new linear module, the Multi-scale frequency domain filter (MFDF), is designed to capture global information from the frequency domain. The high order multi-scale relationship is extracted by combining the depthwise atrous separable convolution with the frequency domain filter. Finally, a channel attention called Axial selection channel attention (ASCA) is redesigned to enhance the network’s ability to model feature channel interrelationships. Further, we design a novel frequency domain medical image segmentation baseline method FDFUNet based on the above modules. We conduct extensive experiments on five publicly available medical image datasets and demonstrate that the present method has stronger segmentation performance as well as generalization ability compared to other state-of-the-art baseline methods.

DS-MSFF-Net: Dual-path self-attention multi-scale feature fusion network for CT image segmentation

DS-MSFF-Net: Dual-path self-attention multi-scale feature fusion network for CT image segmentation

PIS-Net: Efficient Medical Image Segmentation Network with Multivariate Downsampling for Point-of-Care.

Recently, with more portable diagnostic devices being moved to people anywhere, point-of-care (PoC) imaging has become more convenient and more popular than the traditional "bed imaging". Instant image segmentation, as an important technology of computer vision, is receiving more and more attention in PoC diagnosis. However, the image distortion caused by image preprocessing and the low resolution of medical images extracted by PoC devices are urgent problems that need to be solved. Moreover, more efficient feature representation is necessary in the design of instant image segmentation. In this paper, a new feature representation considering the relationships among local features with minimal parameters and a lower computational complexity is proposed. Since a feature window sliding along a diagonal can capture more pluralistic features, a Diagonal-Axial Multi-Layer Perceptron is designed to obtain the global correlation among local features for a more comprehensive feature representation. Additionally, a new multi-scale feature fusion is proposed to integrate nonlinear features with linear ones to obtain a more precise feature representation. Richer features are figured out. In order to improve the generalization of the models, a dynamic residual spatial pyramid pooling based on various receptive fields is constructed according to different sizes of images, which alleviates the influence of image distortion. The experimental results show that the proposed strategy has better performance on instant image segmentation. Notably, it yields an average improvement of 1.31% in Dice than existing strategies on the BUSI, ISIC2018 and MoNuSeg datasets.

Medical image super-resolution reconstruction: A comprehensive investigation of Generative Adversarial Networks

Medical images play a crucial role in modern healthcare diagnostics and treatment. However, many medical images suffer from limitations in resolution, potentially impeding a comprehensive understanding of a patient's condition by healthcare professionals. This comprehensive review delves into the applications of Generative Adversarial Networks (GANs) in medical image super-resolution reconstruction to address this challenge. In the Methods section, this paper first focused on the direction of medical image classification, including cell classification of histopathological images and synthetic data enhancement using GANs to improve liver lesion classification. Subsequently, this paper focused on the direction of medical image segmentation, looking into the use of Structure-Corrected Adversarial Networks (SCAN) for organ segmentation in chest radiographs and Deep Adversarial Networks for biomedical image segmentation using unannotated images. In the Applications and Discussion section, this paper thoroughly examined the current progress of GANs in telemedicine diagnosis and disease state generation and prediction. This paper emphasized the significant potential of GAN technology in telemedicine while outlining the current constraints and challenges. Furthermore, this paper highlighted the prospects of GANs in medical image super-resolution reconstruction and how they affect the discipline of medical imaging. This comprehensive review consolidates the latest research findings on GANs in medical image super-resolution reconstruction, underscoring their importance in the realm of healthcare. By critically analysing existing literature, this paper provides valuable insights for medical image analysts are researchers while inspiring future research directions and innovations.

A Comprehensive Brain MRI Image Segmentation System Based on Contourlet Transform and Deep Neural Networks

Brain tumors are one of the deadliest types of cancer. Rapid and accurate identification of brain tumors, followed by appropriate surgical intervention or chemotherapy, increases the probability of survival. Accurate determination of brain tumors in MRI scans determines the exact location of surgical intervention or chemotherapy. However, this accurate segmentation of brain tumors, due to their diverse morphologies in MRI scans, poses challenges that require significant expertise and accuracy in image interpretation. Despite significant advances in this field, there are several barriers to proper data collection, particularly in the medical sciences, due to concerns about the confidentiality of patient information. However, research papers for learning systems and proposed networks often rely on standardized datasets because a specific approach is unavailable. This system combines unsupervised learning in the adversarial generative network component with supervised learning in segmentation networks. The system is fully automated and can be applied to tumor segmentation on various datasets, including those with sparse data. In order to improve the learning process, the brain MRI segmentation network is trained using a generative adversarial network to increase the number of images. The U-Net model was employed during the segmentation step to combine the remaining blocks efficiently. Contourlet transform produces the ground truth for each MRI image obtained from the adversarial generator network and the original images in the processing and mask preparation phase. On the part of the adversarial generator network, high-quality images are produced, the results of which are similar to the histogram of the original images. Finally, this system improves the image segmentation performance by combining the remaining blocks with the U-net network. Segmentation is evaluated using brain magnetic resonance images obtained from Istanbul Medipol Hospital. The results show that the proposed method and image segmentation network, which incorporates several criteria, such as the DICE criterion of 0.9434, can be effectively used in any dataset as a fully automatic system for segmenting different brain MRI images.

Integrated Design of Building Environment Based on Image Segmentation and Retrieval Technology

Existing models still exhibit a deficiency in capturing more detailed contextual information when processing architectural images. This paper introduces a model for architectural image segmentation and retrieval based on an image segmentation network. Primarily, spatial attention is incorporated into the U-Net segmentation network to enhance the extraction of image features. Subsequently, a dual-path attention mechanism is integrated into the U-Net backbone network, facilitating the seamless integration of information across different spaces and scales. Experimental results showcase the superior performance of the proposed model on the test set, with average dice coefficient, accuracy, and recall reaching 94.67%, 95.61%, and 97.88%, respectively, outperforming comparative models. The proposed model can enhance the U-Net network's capability to identify targets within feature maps. The amalgamation of image segmentation networks and attention mechanisms in artificial intelligence technology enables precise segmentation and retrieval of architectural images.

DSNet: A dynamic squeeze network for real-time weld seam image segmentation

The image noise generated by the welding process, such as arc light, splash, and smoke, brings significant challenges for the laser vision sensor-based welding robot to locate the weld seam and accurately conduct automatic welding. Currently, deep learning-based approaches surpass traditional methods in flexibility and robustness. However, their significant computational cost leads to a mismatch with the real-time requirement of automated welding. In this paper, we propose an efficient hybrid architecture of Convolutional Neural Network (CNN) and transformer, referred to as Dynamic Squeeze Network (DSNet), for real-time weld seam segmentation. More precisely, a lightweight segmentation framework is developed to fully leverage the advantages of the transformer structure without significantly increasing computational overhead. In this respect, an efficient encoder, which aims to increase its features diversity, has been designed and resulted in substantial improvement of encoding performance. Moreover, we propose a plug-and-play lightweight attention module that generates more effective attention weights by exploiting statistical information of weld seam data and introducing linear priors. Extensive experiments on weld seam images using NVIDIA GTX 1050Ti show that our approach reduces the number of parameters by 54x, decreases computational complexity by 34x, and improves inference speed by 33x compared to the baseline method TransUNet. DSNet achieves superior accuracy (78.01% IoU, 87.64% Dice) and speed performance (100 FPS) with lower model complexity and computational burden than most state-of-the-art methods. The code is available at https://github.com/hackerschen/DSNet.

MpMsCFMA-Net: Multi-path Multi-scale Context Feature Mixup and Aggregation Network for medical image segmentation

Automatic and accurate medical image segmentation is a crucial step for clinical diagnosis and treatment planning of diseases. The advanced convolutional neural network (CNN) approaches based on the encoder–decoder structure have achieved state-of-the-art performances in many different medical image segmentation tasks. However, existing networks have insufficient capability of extracting the context information in each encoding stage, so they cannot effectively perceive multi-scale objects in images. In addition, the continuous down-sampling and convolution operations in the encoding stage lead to much loss of the detailed information, resulting in poor segmentation performance. In this paper, we propose a Multi-path Multi-scale Context Feature MixUp and Aggregation Network (named MpMsCFMA-Net) which fuses and aggregates multi-path features with multi-scale context information to address these issues. Based on the encoder–decoder structure, we first design the encoder to encode the semantic and detailed information of input images and introduce multi-scale context extraction module in each encoding stage. Furthermore, we design multiple features mixup module between the encoder and the decoder, aiming at providing different levels of global context information for the decoder by reconstructing skip-connection. Finally, we introduce the decoder with deeper features aggregation to better fuse multi-scale context information across layers. Experimental results on four public medical image datasets confirm that our proposed network achieves promising results and outperforms other state-of-the-art methods in most of evaluation metrics. The source code will be publicly available at https://github.com/tricksterANDthug/MpMsCFMA-Net.

PFD-Net: Pyramid Fourier Deformable Network for medical image segmentation

Medical image segmentation is crucial for accurately locating lesion regions and assisting doctors in diagnosis. However, most existing methods fail to effectively utilize both local details and global semantic information in medical image segmentation, resulting in the inability to effectively capture fine-grained content such as small targets and irregular boundaries. To address this issue, we propose a novel Pyramid Fourier Deformable Network (PFD-Net) for medical image segmentation, which leverages the strengths of CNN and Transformer. The PFD-Net first utilizes PVTv2-based Transformer as the primary encoder to capture global information and further enhances both local and global feature representations with the Fast Fourier Convolution Residual (FFCR) module. Moreover, PFD-Net further proposes the Dilated Deformable Refinement (DDR) module to enhance the model’s capacity to comprehend global semantic structures of shape-diverse targets and their irregular boundaries. Lastly, Cross-Level Fusion Block with deformable convolution (CLFB) is proposed to combine the decoded feature maps from the final Residual Decoder Block (DDR) with local features from the CNN auxiliary encoder branch, improving the network’s ability to perceive targets resembling the surrounding structures. Extensive experiments were conducted on nine publicly medical image datasets for five types of segmentation tasks including polyp, abdominal, cardiac, gland cells and nuclei. The qualitative and quantitative results demonstrate that PFD-Net outperforms existing state-of-the-art methods in various evaluation metrics, and achieves the highest performance of mDice with the value of 0.826 on the most challenging dataset (ETIS), which is 1.8% improvement compared to the previous best-performing HSNet and 3.6% improvement compared to the next-best PVT-CASCADE. Codes are available at https://github.com/ChaorongYang/PFD-Net.

STCNet: Alternating CNN and improved transformer network for COVID-19 CT image segmentation

Since the emergence of the Corona Virus Disease in 2019 (COVID-19), it has become a serious health problem affecting the human respiratory system. At present, automatic segmentation of lung infection areas from Computed Tomography has been playing a crucial role in the diagnosis of this disease because of its ability to perform pathological studies based on the lung infection areas. However, due to the lung infection areas scattered distribution, the existing segmentation methods generally have the problems of missing and incomplete segmentation. The Convolutional Neural Network (CNN)-based approaches generally lack the ability to model explicit long-range relation, and the transformer-based methods are not conducive to capturing the detailed boundaries of infected areas. Whereas the infected regions of the coronavirus images are scattered and boundary information plays an important role, both the boundaries and the global infected areas need to be taken into account. Therefore, we propose a novel coronavirus image segmentation network alternately using Swin transformer and CNN (STCNet). Firstly, to enable network to capture richer features, the ReSwin transformer block is proposed and added after each level of convolution block in the encoder-decoder. Secondly, to effectively retain the infected areas boundary information, the skip connection cross attention module is used to provide spatial information to each decoder. And through the fine-tuned scale-aware pyramid fusion module to fuse multi-scale context information. Experimental results show that STCNet at can achieve state-of-the-art performance on two coronavirus segmentation datasets, with Dice achieves 79.92 % and 82.78 %, respectively. Our code is available at https://github.com/sineagles/STCNet.

US2Mask: Image-to-mask generation learning via a conditional GAN for cardiac ultrasound image segmentation

Cardiac ultrasound (US) image segmentation is vital for evaluating clinical indices, but it often demands a large dataset and expert annotations, resulting in high costs for deep learning algorithms. To address this, our study presents a framework utilizing artificial intelligence generation technology to produce multi-class RGB masks for cardiac US image segmentation. The proposed approach directly performs semantic segmentation of the heart’s main structures in US images from various scanning modes. Additionally, we introduce a novel learning approach based on conditional generative adversarial networks (CGAN) for cardiac US image segmentation, incorporating a conditional input and paired RGB masks. Experimental results from three cardiac US image datasets with diverse scan modes demonstrate that our approach outperforms several state-of-the-art models, showcasing improvements in five commonly used segmentation metrics, with lower noise sensitivity. Source code is available at https://github.com/energy588/US2mask.

Scalable semantic 3D mapping of coral reefs with deep learning

Abstract Coral reefs are among the most diverse ecosystems on our planet, and essential to the livelihood of hundreds of millions of people who depend on them for food security, income from tourism and coastal protection. Unfortunately, most coral reefs are existentially threatened by global climate change and local anthropogenic pressures. To better understand the dynamics underlying deterioration of reefs, monitoring at high spatial and temporal resolution is key. However, conventional monitoring methods for quantifying coral cover and species abundance are limited in scale due to the extensive manual labor required. Although computer vision tools have been employed to aid in this process, in particular structure‐from‐motion (SfM) photogrammetry for 3D mapping and deep neural networks for image segmentation, analysis of the data products creates a bottleneck, effectively limiting their scalability. This paper presents a new paradigm for mapping underwater environments from ego‐motion video, unifying 3D mapping systems that use machine learning to adapt to challenging conditions under water, combined with a modern approach for semantic segmentation of images. The method is exemplified on coral reefs in the northern Gulf of Aqaba, Red Sea, demonstrating high‐precision 3D semantic mapping at unprecedented scale with significantly reduced required labor costs: given a trained model, a 100 m video transect acquired within 5 min of diving with a cheap consumer‐grade camera can be fully automatically transformed into a semantic point cloud within 5 min. We demonstrate the spatial accuracy of our method and the semantic segmentation performance (of at least 80% total accuracy), and publish a large dataset of ego‐motion videos from the northern Gulf of Aqaba, along with a dataset of video frames annotated for dense semantic segmentation of benthic classes. Our approach significantly scales up coral reef monitoring by taking a leap towards fully automatic analysis of video transects. The method advances coral reef transects by reducing the labor, equipment, logistics, and computing cost. This can help to inform conservation policies more efficiently. The underlying computational method of learning‐based Structure‐from‐Motion has broad implications for fast low‐cost mapping of underwater environments other than coral reefs.

MCCSeg: Morphological embedding causal constraint network for medical image segmentation

Medical image multi-object segmentation aims to accurately extract each object that is great significant for the medical image analysis. Although several methods based on deep learning have been widely developed, it is still challenging due to the similarities of different objects, the significant morphological variability, data heterogeneity, and the unbalanced distribution of samples. To address the above issues, we propose a novel Morphological Embedding Causal Constraint segmentation network (MCCSeg) for medical image multi-object segmentation. We build a hybrid CNN-Transformer encoder module to obtain local and global features. Specifically, a Causal Constraint Module (CCM) is proposed to decorrelate features by sample reweighting, which utilizes weight allocation to train the weight of training samples. The Random Fourier Features (RFF) is employed to solve nonlinear dependency problems and remove correlations between features to mitigate the influence of spurious correlations. The morphological guidance module (MG) is designed to extract the boundary as prior morphological information for enhancing feature representation, which is concatenated into the deep supervision module in the decoder for further optimizing feature extraction. The experiments demonstrate that MCCSeg outperforms other state-of-the-art methods, achieving improvements of 3.76% and 5.41% in terms of DICE and HD95 scores on Synapse dataset, respectively. The ablation experiments further verify the effectiveness of MCCSeg in improving generalization.

SC-Net: Symmetrical conical network for colorectal pathology image segmentation

Background and ObjectiveImage segmentation of histopathology of colorectal cancer is a core task of computer aided medical image diagnosis system. Existing convolutional neural networks generally extract multi-scale information in linear flow structures by inserting multi-branch modules, which is difficult to extract heterogeneous semantic information under multi-level and different receptive field and tough to establish context dependency among different receptive field features. MethodsTo address these issues, we propose a symmetric spiral progressive feature fusion encoder-decoder network called the Symmetric Conical Network (SC-Net). First, we design a Multi-scale Feature Extraction Block (MFEB) matching with the Symmetric Conical Network to obtain multi-branch heterogeneous semantic information under different receptive fields, so as to enrich the diversity of extracted feature information. The encoder is composed of MFEB through spiral and multi-branch arrangement to enhance context dependence between different information flow. Secondly, the information loss of contour, color and others in high-level semantic information through causally stacking MFEB, the Feature Mapping Layer (FML) is designed to map low-level features to high-level semantic features along the down-sampling branch and solve the problem of insufficient global feature extraction in deep levels. ResultsThe SC-Net was evaluated on our self-constructed colorectal cancer dataset, a publicly available breast cancer dataset and a polyp dataset. The results revealed that the mDice of segmentation reached 0.8611, 0.7259 and 0.7144. We compare our model with the state-of-art semantic segmentation UNet++, PSPNet, Attention U-Net, R2U-Net and other advanced segmentation networks. The experimental results demonstrate that we achieve the most advanced performance. ConclusionsThe results indicate that the proposed SC-Net excels in segmenting H&E stained pathology images, effectively preserving morphological features and spatial information even in scenarios with weak texture, poor contrast, and variations in appearance.

MP-Net: A Multi-Center Privacy-Preserving Network for Medical Image Segmentation.

In this paper, we present the Multi-Center Privacy-Preserving Network (MP-Net), a novel framework designed for secure medical image segmentation in multi-center collaborations. Our methodology offers a new approach to multi-center collaborative learning, capable of reducing the volume of data transmission and enhancing data privacy protection. Unlike federated learning, which requires the transmission of model data between the central server and local servers in each round, our method only necessitates a single transfer of encrypted data. The proposed MP-Net comprises a three-layer model, consisting of encryption, segmentation, and decryption networks. We encrypt the image data into ciphertext using an encryption network and introduce an improved U-Net for image ciphertext segmentation. Finally, the segmentation mask is obtained through a decryption network. This architecture enables ciphertext-based image segmentation through computable image encryption. We evaluate the effectiveness of our approach on three datasets, including two cardiac MRI datasets and a CTPA dataset. Our results demonstrate that the MP-Net can securely utilize data from multiple centers to establish a more robust and information-rich segmentation model.

Entropy Kernel Graph Cut Feature Space Enhancement with SqueezeNet Deep Neural Network for Textural Image Segmentation

Recently, image segmentation based on graph cut methods has shown remarkable performance on a set of image data. Although the kernel graph cut method provides good performance, its performance is highly dependent on the data mapping to the transformation space and image features. The entropy-based kernel graph cut method is suitable for segmentation of textured images. Nonetheless, its segmentation quality remains significantly contingent on the accuracy and richness of feature space representation and kernel centers. This paper introduces an entropy-based kernel graph cut method, which leverages the discriminative feature space extracted from SqueezeNet, a deep neural network. The fusion of SqueezeNet’s features enriches the segmentation process by capturing high-level semantic information. Moreover, the extraction of kernel centers is refined through a weighted k-means approach, contributing further to the segmentation’s precision and effectiveness. The proposed method, while exploiting the benefits of suitable computational load of graph cut methods, will be a suitable alternative for segmenting textured images. Laboratory results have been taken on a set of well-known datasets that include textured shapes in order to evaluate the efficiency of the algorithm compared to other well-known methods in the field of kernel graph cut.