Network For Image Segmentation Research Articles

RCEAU-Net: Cascade Multi-Scale Convolution and Attention-Mechanism-Based Network for Laser Beam Target Image Segmentation with Complex Background in Coal Mine.

Accurate and reliable pose estimation of boom-type roadheaders is the key to the forming quality of the tunneling face in coal mines, which is of great importance to improve tunneling efficiency and ensure the safety of coal mine production. The multi-laser-beam target-based visual localization method is an effective way to realize accurate and reliable pose estimation of a roadheader body. However, the complex background interference in coal mines brings great challenges to the stable and accurate segmentation and extraction of laser beam features, which has become the main problem faced by the long-distance visual positioning method of underground equipment. In this paper, a semantic segmentation network for underground laser beams in coal mines, RCEAU-Net, is proposed based on U-Net. The network introduces residual connections in the convolution of the encoder and decoder parts, which effectively fuses the underlying feature information and improves the gradient circulation performance of the network. At the same time, by introducing cascade multi-scale convolution in the skipping connection section, which compensates for the lack of contextual semantic information in U-Net and improves the segmentation effect of the network model on tiny laser beams at long distance. Finally, the introduction of an efficient multi-scale attention module with cross-spatial learning in the encoder enhances the feature extraction capability of the network. Furthermore, the laser beam target dataset (LBTD) is constructed based on laser beam target images collected from several coal mines, and the proposed RCEAU-Net model is then tested and verified. The experimental results show that, compared with the original U-Net, RCEAU-Net can ensure the real-time performance of laser beam segmentation while increasing the Accuracy by 0.19%, Precision by 2.53%, Recall by 22.01%, and Intersection and Union Ratio by 8.48%, which can meet the requirements of multi-laser-beam feature segmentation and extraction under complex backgrounds in coal mines, so as to further ensure the accuracy and stability of long-distance visual positioning for boom-type roadheaders and ensure the safe production in the working face.

UCTNet: Uncertainty-guided CNN-Transformer hybrid networks for medical image segmentation

Transformer, born for long-range dependency establishment, has been widely studied as a complementary of convolutional neural networks (CNNs) in medical image segmentation. However, existing CNN-Transformer hybrid approaches simply pursue implicit feature fusion without considering their underlying functional overlap. Medical images typically follow stable anatomical structures, making convolution capable of handling most segmentation targets. Without differentiation, enforcing transformers to operate self-attention for all image patches would result in severe redundancy, hindering global feature extraction. In this paper, we propose a simple yet effective hybrid network named UCTNet where transformers only focus on establishing global dependency for CNN’s unreliable regions predicted through uncertainty estimation. In this way, CNN and transformer are explicitly fused to minimize functional overlap. More importantly, with fewer regions to handle, UCTNet is of better convergence to learn more robust feature representations for hard examples. Extensive experiments on publicly-available datasets demonstrate the superiority of UCTNet against the state-of-the-art approaches, achieving 89.44%, 92.91%, and 91.15% in Dice similarity coefficient on Synapse, ACDC, and ISIC2018 respectively. Furthermore, such a CNN-Transformer hybrid strategy is highly extendable to other frameworks without introducing additional computational burdens. Code is available at https://github.com/innocence0206/UCTNet.

Self-supervised multi-magnification feature enhancement for segmentation of hepatocellular carcinoma region in pathological images

Accurate segmentation of hepatocellular carcinoma regions is a critical step in surgical evaluation of whole slide pathological images. Recently, emerging multi-magnification learning-based methods have shown promise with evaluation of whole slide images. However, traditional multi-magnification learning segmentation models focus on uniformity and fail to effectively exploit different magnification information. The purpose of this study is to develop a novel multi-magnification learning segmentation model by effectively utilizing diverse magnification information. Therefore, we developed a novel multi-magnification feature self-enhancement network to extract magnification-specific features from multi-magnification images via self-supervised learning without extra supervision. Specifically, the proposed network enhances feature information via a super-resolution module based on self-supervision to transfer the representative information into a segmentation encoder, which takes full advantage of the pyramid storage characteristic of whole slide images. Moreover, a multi-scale feature fusion module based on the attention mechanism is used to fuse pretrained multi-scale features through a gated unit block, which is designed to fuse features from generation task into and segmentation task. We evaluated this method using the hepatocellular carcinoma data set from The Cancer Genome Atlas with cross-validation and achieved a 0.829 Dice similarity coefficient, nearly a 3% improvement over state-of-the-art models used for segmentation. To the best of our knowledge, this is the first study of the application of super-resolution as a self-enhancement network for histopathological image segmentation via self-supervised pretext learning. The code will be available at https://github.com/SH-Diao123/Self-Supervised-Multi-magnification-Segmentation.

Superpixel-Based Graph Convolutional Network for UAV Forest Fire Image Segmentation

Given the escalating frequency and severity of global forest fires, it is imperative to develop advanced detection and segmentation technologies to mitigate their impact. To address the challenges of these technologies, the development of deep learning-based forest fire surveillance has significantly accelerated. Nevertheless, the integration of graph convolutional networks (GCNs) in forest fire detection remains relatively underexplored. In this context, we introduce a novel superpixel-based graph convolutional network (SCGCN) for forest fire image segmentation. Our proposed method utilizes superpixels to transform images into a graph structure, thereby reinterpreting the image segmentation challenge as a node classification task. Additionally, we transition the spatial graph convolution operation to a GraphSAGE graph convolution mechanism, mitigating the class imbalance issue and enhancing the network’s versatility. We incorporate an innovative loss function to contend with the inconsistencies in pixel dimensions within superpixel clusters. The efficacy of our technique is validated on two different forest fire datasets, demonstrating superior performance compared to four alternative segmentation methodologies.

STCS-Net: a medical image segmentation network that fully utilizes multi-scale information

In recent years, significant progress has been made in the field of medical image segmentation through the application of deep learning and neural networks. Numerous studies have focused on optimizing encoders to extract more comprehensive key information. However, the importance of decoders in directly influencing the final output of images cannot be overstated. The ability of decoders to effectively leverage diverse information and further refine crucial details is of paramount importance. This paper proposes a medical image segmentation architecture named STCS-Net. The designed decoder in STCS-Net facilitates multi-scale filtering and correction of information from the encoder, thereby enhancing the accuracy of extracting vital features. Additionally, an information enhancement module is introduced in skip connections to highlight essential features and improve the inter-layer information interaction capabilities. Comprehensive evaluations on the ISIC2016, ISIC2018, and Lung datasets validate the superiority of STCS-Net across different scenarios. Experimental results demonstrate the outstanding performance of STCS-Net on all three datasets. Comparative experiments highlight the advantages of our proposed network in terms of accuracy and parameter efficiency. Ablation studies confirm the effectiveness of the introduced decoder and skip connection module. This research introduces a novel approach to the field of medical image segmentation, providing new perspectives and solutions for future developments in medical image processing and analysis.

Image Semantic Segmentation Approach for Studying Human Behavior on Image Data

Image semantic segmentation is an essential technique for studying human behavior through image data. This paper proposes an image semantic segmentation method for human behavior research. Firstly, an end-to-end convolutional neural network architecture is proposed, which consists of a depth-separable jump-connected fully convolutional network and a conditional random field network; then jump-connected convolution is used to classify each pixel in the image, and an image semantic segmentation method based on convolutional neural network is proposed; and then a conditional random field network is used to improve the effect of image segmentation of human behavior and a linear modeling and nonlinear modeling method based on the semantic segmentation of conditional random field image is proposed. Finally, using the proposed image segmentation network, the input entrepreneurial image data is semantically segmented to obtain the contour features of the person; and the segmentation of the images in the medical field. The experimental results show that the image semantic segmentation method is effective. It is a new way to use image data to study human behavior and can be extended to other research areas.

PAMSNet: A medical image segmentation network based on spatial pyramid and attention mechanism

The image segmentation of diseases can help clinical diagnosis and treatment in medical image analysis. Due to the complexity of lesion features (e.g., size, location, and morphology) and the high similarity between the background and the target area in medical images, semantic features are difficult to extract completely. To tackle these problems, we propose a novel medical image segmentation network PAMSNet based on the spatial pyramid and attention mechanism. By using efficient pyramid attention and channel spatial attention modules, the proposed method fuses the extracted multi-scale spatial information with the local features extracted by the encoder to supplement the image details. In addition, the Spatial Pyramid-Coordinate Attention (SPCA) module is introduced in the bottleneck layer to obtain larger receptive field information and enhance feature extraction. We conducted qualitative and quantitative evaluations on four public datasets, including ISIC2018, Lung segmentation, Kvasir-SEG, and ISLES2022. The segmentation accuracy of DSC was 87.86%, 98.18%, 82.43%, and 87.37%, respectively. The ablation study of each part of PAMSNet proves the validity of each component, and the comparison with state-of-the-art methods on different indicators proves the predominance of the network.

SDMI-Net: Spatially Dependent Mutual Information Network for semi-supervised medical image segmentation

Semi-supervised medical image segmentation strives to polish deep models with a small amount of labeled data and a large amount of unlabeled data. The efficiency of most semi-supervised medical image segmentation methods based on voxel-level consistency learning is affected by low-confidence voxels. In addition, voxel-level consistency learning fails to consider the spatial correlation between neighboring voxels. To encourage reliable voxel-level consistent learning, we propose a dual-teacher affine consistent uncertainty estimation method to filter out some voxels with high uncertainty. Moreover, we design the spatially dependent mutual information module, which enhances the spatial dependence between neighboring voxels by maximizing the mutual information between the local voxel blocks predicted from the dual-teacher models and the student model, enabling consistent learning at the block level. On two benchmark medical image segmentation datasets, including the Left Atrial Segmentation Challenge dataset and the BraTS-2019 dataset, our method achieves state-of-the-art performance in both quantitative and qualitative aspects.

Medical image segmentation network based on multi-scale frequency domain filter

With the development of deep learning, medical image segmentation in computer-aided diagnosis has become a research hotspot. Recently, UNet and its variants have become the most powerful medical image segmentation methods. However, these methods suffer from (1) insufficient sensing field and insufficient depth; (2) computational nonlinearity and redundancy of channel features; and (3) ignoring the interrelationships among feature channels. These problems lead to poor network segmentation performance and weak generalization ability. Therefore, first of all, we propose an effective replacement scheme of UNet base block, Double residual depthwise atrous convolution (DRDAC) block, to effectively improve the deficiency of receptive field and depth. Secondly, a new linear module, the Multi-scale frequency domain filter (MFDF), is designed to capture global information from the frequency domain. The high order multi-scale relationship is extracted by combining the depthwise atrous separable convolution with the frequency domain filter. Finally, a channel attention called Axial selection channel attention (ASCA) is redesigned to enhance the network’s ability to model feature channel interrelationships. Further, we design a novel frequency domain medical image segmentation baseline method FDFUNet based on the above modules. We conduct extensive experiments on five publicly available medical image datasets and demonstrate that the present method has stronger segmentation performance as well as generalization ability compared to other state-of-the-art baseline methods.

DS-MSFF-Net: Dual-path self-attention multi-scale feature fusion network for CT image segmentation

DS-MSFF-Net: Dual-path self-attention multi-scale feature fusion network for CT image segmentation

PIS-Net: Efficient Medical Image Segmentation Network with Multivariate Downsampling for Point-of-Care.

Recently, with more portable diagnostic devices being moved to people anywhere, point-of-care (PoC) imaging has become more convenient and more popular than the traditional "bed imaging". Instant image segmentation, as an important technology of computer vision, is receiving more and more attention in PoC diagnosis. However, the image distortion caused by image preprocessing and the low resolution of medical images extracted by PoC devices are urgent problems that need to be solved. Moreover, more efficient feature representation is necessary in the design of instant image segmentation. In this paper, a new feature representation considering the relationships among local features with minimal parameters and a lower computational complexity is proposed. Since a feature window sliding along a diagonal can capture more pluralistic features, a Diagonal-Axial Multi-Layer Perceptron is designed to obtain the global correlation among local features for a more comprehensive feature representation. Additionally, a new multi-scale feature fusion is proposed to integrate nonlinear features with linear ones to obtain a more precise feature representation. Richer features are figured out. In order to improve the generalization of the models, a dynamic residual spatial pyramid pooling based on various receptive fields is constructed according to different sizes of images, which alleviates the influence of image distortion. The experimental results show that the proposed strategy has better performance on instant image segmentation. Notably, it yields an average improvement of 1.31% in Dice than existing strategies on the BUSI, ISIC2018 and MoNuSeg datasets.

Medical image super-resolution reconstruction: A comprehensive investigation of Generative Adversarial Networks

Medical images play a crucial role in modern healthcare diagnostics and treatment. However, many medical images suffer from limitations in resolution, potentially impeding a comprehensive understanding of a patient's condition by healthcare professionals. This comprehensive review delves into the applications of Generative Adversarial Networks (GANs) in medical image super-resolution reconstruction to address this challenge. In the Methods section, this paper first focused on the direction of medical image classification, including cell classification of histopathological images and synthetic data enhancement using GANs to improve liver lesion classification. Subsequently, this paper focused on the direction of medical image segmentation, looking into the use of Structure-Corrected Adversarial Networks (SCAN) for organ segmentation in chest radiographs and Deep Adversarial Networks for biomedical image segmentation using unannotated images. In the Applications and Discussion section, this paper thoroughly examined the current progress of GANs in telemedicine diagnosis and disease state generation and prediction. This paper emphasized the significant potential of GAN technology in telemedicine while outlining the current constraints and challenges. Furthermore, this paper highlighted the prospects of GANs in medical image super-resolution reconstruction and how they affect the discipline of medical imaging. This comprehensive review consolidates the latest research findings on GANs in medical image super-resolution reconstruction, underscoring their importance in the realm of healthcare. By critically analysing existing literature, this paper provides valuable insights for medical image analysts are researchers while inspiring future research directions and innovations.

A Comprehensive Brain MRI Image Segmentation System Based on Contourlet Transform and Deep Neural Networks

Brain tumors are one of the deadliest types of cancer. Rapid and accurate identification of brain tumors, followed by appropriate surgical intervention or chemotherapy, increases the probability of survival. Accurate determination of brain tumors in MRI scans determines the exact location of surgical intervention or chemotherapy. However, this accurate segmentation of brain tumors, due to their diverse morphologies in MRI scans, poses challenges that require significant expertise and accuracy in image interpretation. Despite significant advances in this field, there are several barriers to proper data collection, particularly in the medical sciences, due to concerns about the confidentiality of patient information. However, research papers for learning systems and proposed networks often rely on standardized datasets because a specific approach is unavailable. This system combines unsupervised learning in the adversarial generative network component with supervised learning in segmentation networks. The system is fully automated and can be applied to tumor segmentation on various datasets, including those with sparse data. In order to improve the learning process, the brain MRI segmentation network is trained using a generative adversarial network to increase the number of images. The U-Net model was employed during the segmentation step to combine the remaining blocks efficiently. Contourlet transform produces the ground truth for each MRI image obtained from the adversarial generator network and the original images in the processing and mask preparation phase. On the part of the adversarial generator network, high-quality images are produced, the results of which are similar to the histogram of the original images. Finally, this system improves the image segmentation performance by combining the remaining blocks with the U-net network. Segmentation is evaluated using brain magnetic resonance images obtained from Istanbul Medipol Hospital. The results show that the proposed method and image segmentation network, which incorporates several criteria, such as the DICE criterion of 0.9434, can be effectively used in any dataset as a fully automatic system for segmenting different brain MRI images.

Integrated Design of Building Environment Based on Image Segmentation and Retrieval Technology

Existing models still exhibit a deficiency in capturing more detailed contextual information when processing architectural images. This paper introduces a model for architectural image segmentation and retrieval based on an image segmentation network. Primarily, spatial attention is incorporated into the U-Net segmentation network to enhance the extraction of image features. Subsequently, a dual-path attention mechanism is integrated into the U-Net backbone network, facilitating the seamless integration of information across different spaces and scales. Experimental results showcase the superior performance of the proposed model on the test set, with average dice coefficient, accuracy, and recall reaching 94.67%, 95.61%, and 97.88%, respectively, outperforming comparative models. The proposed model can enhance the U-Net network's capability to identify targets within feature maps. The amalgamation of image segmentation networks and attention mechanisms in artificial intelligence technology enables precise segmentation and retrieval of architectural images.

DSNet: A dynamic squeeze network for real-time weld seam image segmentation

The image noise generated by the welding process, such as arc light, splash, and smoke, brings significant challenges for the laser vision sensor-based welding robot to locate the weld seam and accurately conduct automatic welding. Currently, deep learning-based approaches surpass traditional methods in flexibility and robustness. However, their significant computational cost leads to a mismatch with the real-time requirement of automated welding. In this paper, we propose an efficient hybrid architecture of Convolutional Neural Network (CNN) and transformer, referred to as Dynamic Squeeze Network (DSNet), for real-time weld seam segmentation. More precisely, a lightweight segmentation framework is developed to fully leverage the advantages of the transformer structure without significantly increasing computational overhead. In this respect, an efficient encoder, which aims to increase its features diversity, has been designed and resulted in substantial improvement of encoding performance. Moreover, we propose a plug-and-play lightweight attention module that generates more effective attention weights by exploiting statistical information of weld seam data and introducing linear priors. Extensive experiments on weld seam images using NVIDIA GTX 1050Ti show that our approach reduces the number of parameters by 54x, decreases computational complexity by 34x, and improves inference speed by 33x compared to the baseline method TransUNet. DSNet achieves superior accuracy (78.01% IoU, 87.64% Dice) and speed performance (100 FPS) with lower model complexity and computational burden than most state-of-the-art methods. The code is available at https://github.com/hackerschen/DSNet.

MpMsCFMA-Net: Multi-path Multi-scale Context Feature Mixup and Aggregation Network for medical image segmentation

Automatic and accurate medical image segmentation is a crucial step for clinical diagnosis and treatment planning of diseases. The advanced convolutional neural network (CNN) approaches based on the encoder–decoder structure have achieved state-of-the-art performances in many different medical image segmentation tasks. However, existing networks have insufficient capability of extracting the context information in each encoding stage, so they cannot effectively perceive multi-scale objects in images. In addition, the continuous down-sampling and convolution operations in the encoding stage lead to much loss of the detailed information, resulting in poor segmentation performance. In this paper, we propose a Multi-path Multi-scale Context Feature MixUp and Aggregation Network (named MpMsCFMA-Net) which fuses and aggregates multi-path features with multi-scale context information to address these issues. Based on the encoder–decoder structure, we first design the encoder to encode the semantic and detailed information of input images and introduce multi-scale context extraction module in each encoding stage. Furthermore, we design multiple features mixup module between the encoder and the decoder, aiming at providing different levels of global context information for the decoder by reconstructing skip-connection. Finally, we introduce the decoder with deeper features aggregation to better fuse multi-scale context information across layers. Experimental results on four public medical image datasets confirm that our proposed network achieves promising results and outperforms other state-of-the-art methods in most of evaluation metrics. The source code will be publicly available at https://github.com/tricksterANDthug/MpMsCFMA-Net.

PFD-Net: Pyramid Fourier Deformable Network for medical image segmentation

Medical image segmentation is crucial for accurately locating lesion regions and assisting doctors in diagnosis. However, most existing methods fail to effectively utilize both local details and global semantic information in medical image segmentation, resulting in the inability to effectively capture fine-grained content such as small targets and irregular boundaries. To address this issue, we propose a novel Pyramid Fourier Deformable Network (PFD-Net) for medical image segmentation, which leverages the strengths of CNN and Transformer. The PFD-Net first utilizes PVTv2-based Transformer as the primary encoder to capture global information and further enhances both local and global feature representations with the Fast Fourier Convolution Residual (FFCR) module. Moreover, PFD-Net further proposes the Dilated Deformable Refinement (DDR) module to enhance the model’s capacity to comprehend global semantic structures of shape-diverse targets and their irregular boundaries. Lastly, Cross-Level Fusion Block with deformable convolution (CLFB) is proposed to combine the decoded feature maps from the final Residual Decoder Block (DDR) with local features from the CNN auxiliary encoder branch, improving the network’s ability to perceive targets resembling the surrounding structures. Extensive experiments were conducted on nine publicly medical image datasets for five types of segmentation tasks including polyp, abdominal, cardiac, gland cells and nuclei. The qualitative and quantitative results demonstrate that PFD-Net outperforms existing state-of-the-art methods in various evaluation metrics, and achieves the highest performance of mDice with the value of 0.826 on the most challenging dataset (ETIS), which is 1.8% improvement compared to the previous best-performing HSNet and 3.6% improvement compared to the next-best PVT-CASCADE. Codes are available at https://github.com/ChaorongYang/PFD-Net.

STCNet: Alternating CNN and improved transformer network for COVID-19 CT image segmentation

Since the emergence of the Corona Virus Disease in 2019 (COVID-19), it has become a serious health problem affecting the human respiratory system. At present, automatic segmentation of lung infection areas from Computed Tomography has been playing a crucial role in the diagnosis of this disease because of its ability to perform pathological studies based on the lung infection areas. However, due to the lung infection areas scattered distribution, the existing segmentation methods generally have the problems of missing and incomplete segmentation. The Convolutional Neural Network (CNN)-based approaches generally lack the ability to model explicit long-range relation, and the transformer-based methods are not conducive to capturing the detailed boundaries of infected areas. Whereas the infected regions of the coronavirus images are scattered and boundary information plays an important role, both the boundaries and the global infected areas need to be taken into account. Therefore, we propose a novel coronavirus image segmentation network alternately using Swin transformer and CNN (STCNet). Firstly, to enable network to capture richer features, the ReSwin transformer block is proposed and added after each level of convolution block in the encoder-decoder. Secondly, to effectively retain the infected areas boundary information, the skip connection cross attention module is used to provide spatial information to each decoder. And through the fine-tuned scale-aware pyramid fusion module to fuse multi-scale context information. Experimental results show that STCNet at can achieve state-of-the-art performance on two coronavirus segmentation datasets, with Dice achieves 79.92 % and 82.78 %, respectively. Our code is available at https://github.com/sineagles/STCNet.

US2Mask: Image-to-mask generation learning via a conditional GAN for cardiac ultrasound image segmentation

Cardiac ultrasound (US) image segmentation is vital for evaluating clinical indices, but it often demands a large dataset and expert annotations, resulting in high costs for deep learning algorithms. To address this, our study presents a framework utilizing artificial intelligence generation technology to produce multi-class RGB masks for cardiac US image segmentation. The proposed approach directly performs semantic segmentation of the heart’s main structures in US images from various scanning modes. Additionally, we introduce a novel learning approach based on conditional generative adversarial networks (CGAN) for cardiac US image segmentation, incorporating a conditional input and paired RGB masks. Experimental results from three cardiac US image datasets with diverse scan modes demonstrate that our approach outperforms several state-of-the-art models, showcasing improvements in five commonly used segmentation metrics, with lower noise sensitivity. Source code is available at https://github.com/energy588/US2mask.

Scalable semantic 3D mapping of coral reefs with deep learning

Abstract Coral reefs are among the most diverse ecosystems on our planet, and essential to the livelihood of hundreds of millions of people who depend on them for food security, income from tourism and coastal protection. Unfortunately, most coral reefs are existentially threatened by global climate change and local anthropogenic pressures. To better understand the dynamics underlying deterioration of reefs, monitoring at high spatial and temporal resolution is key. However, conventional monitoring methods for quantifying coral cover and species abundance are limited in scale due to the extensive manual labor required. Although computer vision tools have been employed to aid in this process, in particular structure‐from‐motion (SfM) photogrammetry for 3D mapping and deep neural networks for image segmentation, analysis of the data products creates a bottleneck, effectively limiting their scalability. This paper presents a new paradigm for mapping underwater environments from ego‐motion video, unifying 3D mapping systems that use machine learning to adapt to challenging conditions under water, combined with a modern approach for semantic segmentation of images. The method is exemplified on coral reefs in the northern Gulf of Aqaba, Red Sea, demonstrating high‐precision 3D semantic mapping at unprecedented scale with significantly reduced required labor costs: given a trained model, a 100 m video transect acquired within 5 min of diving with a cheap consumer‐grade camera can be fully automatically transformed into a semantic point cloud within 5 min. We demonstrate the spatial accuracy of our method and the semantic segmentation performance (of at least 80% total accuracy), and publish a large dataset of ego‐motion videos from the northern Gulf of Aqaba, along with a dataset of video frames annotated for dense semantic segmentation of benthic classes. Our approach significantly scales up coral reef monitoring by taking a leap towards fully automatic analysis of video transects. The method advances coral reef transects by reducing the labor, equipment, logistics, and computing cost. This can help to inform conservation policies more efficiently. The underlying computational method of learning‐based Structure‐from‐Motion has broad implications for fast low‐cost mapping of underwater environments other than coral reefs.