Semantic Segmentation Task Research Articles

Autonomous Extraction Technology for Aquaculture Ponds in Complex Geological Environments Based on Multispectral Feature Fusion of Medium-Resolution Remote Sensing Imagery

Coastal aquaculture plays a crucial role in global food security and the economic development of coastal regions, but it also causes environmental degradation in coastal ecosystems. Therefore, the automation, accurate extraction, and monitoring of coastal aquaculture areas are crucial for the scientific management of coastal ecological zones. This study proposes a novel deep learning- and attention-based median adaptive fusion U-Net (MAFU-Net) procedure aimed at precisely extracting individually separable aquaculture ponds (ISAPs) from medium-resolution remote sensing imagery. Initially, this study analyzes the spectral differences between aquaculture ponds and interfering objects such as saltwater fields in four typical aquaculture areas along the coast of Liaoning Province, China. It innovatively introduces a difference index for saltwater field aquaculture zones (DIAS) and integrates this index as a new band into remote sensing imagery to increase the expressiveness of features. A median augmented adaptive fusion module (MEA-FM), which adaptively selects channel receptive fields at various scales, integrates the information between channels, and captures multiscale spatial information to achieve improved extraction accuracy, is subsequently designed. Experimental and comparative results reveal that the proposed MAFU-Net method achieves an F1 score of 90.67% and an intersection over union (IoU) of 83.93% on the CHN-LN4-ISAPS-9 dataset, outperforming advanced methods such as U-Net, DeepLabV3+, SegNet, PSPNet, SKNet, UPS-Net, and SegFormer. This study’s results provide accurate data support for the scientific management of aquaculture areas, and the proposed MAFU-Net method provides an effective method for semantic segmentation tasks based on medium-resolution remote sensing images.

The 3D morphometry of Martian craters from HRSC DEMs using a multi-scale semantic segmentation network and morphological analysis

The morphology of impact craters reveals the structure and composition of the Martian surface, especially the subsurface conditions and Martian geological history, which have increasing importance in Mars exploration missions. This work presents a 3D morphometric method for detecting and delineating Martian craters, and a 3D morphological analysis was conducted. Specifically, this work first focused on the segmentation of Martian craters. Based on the segmentation results, clustering of crater instances was then carried out. Finally, with the individual craters that were extracted, a morphological analysis involving the measurements of their diameter, depth, area, RMS height, rim height, circularity, and the statistics thereof was performed. Unlike previous studies, which have mainly used optical images and object detection approaches, this work regards crater extraction as a semantic segmentation task instead of an object detection task to better delineate the precise shape and boundary information. Digital elevation model (DEM) was utilized as primary data to directly obtain 3D information, which was converted into 3D point cloud format and fed to a multi-scale semantic segmentation network. The semantic segmentation results achieved an overall accuracy of 0.932 and mIOU of 0.871 on the test data. We automatically counted 63 craters in Noachis Terra and 40 craters in Terra Cimmeria. The 3D morphological measurements showed that 66% of the impact craters in the first region were larger than 10 km in diameter, while 50% of the impact craters in the second region were larger than 10 km. In both areas, craters could reach a maximum depth of 2000 m. With the proposed method, we can automatically conduct 3D morphological measurements of Martian craters with high efficiency that is improved by 15 times compared with that of manual crater analysis tools. The achieved 3D morphometric results can provide a reference and support for future research on Martian landforms.

Adversarial Unsupervised Domain Adaptation for 3D Semantic Segmentation with 2D Image Fusion of Dense Depth

AbstractUnsupervised domain adaptation (UDA) is increasingly used for 3D point cloud semantic segmentation tasks due to its ability to address the issue of missing labels for new domains. However, most existing unsupervised domain adaptation methods focus only on uni‐modal data and are rarely applied to multi‐modal data. Therefore, we propose a cross‐modal UDA on multi‐modal datasets that contain 3D point clouds and 2D images for 3D Semantic Segmentation. Specifically, we first propose a Dual discriminator‐based Domain Adaptation (Dd‐bDA) module to enhance the adaptability of different domains. Second, given that the robustness of depth information to domain shifts can provide more details for semantic segmentation, we further employ a Dense depth Feature Fusion (DdFF) module to extract image features with rich depth cues. We evaluate our model in four unsupervised domain adaptation scenarios, i.e., dataset‐to‐dataset (A2D2 → SemanticKITTI), Day‐to‐Night, country‐to‐country (USA → Singapore), and synthetic‐to‐real (VirtualKITTI → SemanticKITTI). In all settings, the experimental results achieve significant improvements and surpass state‐of‐the‐art models.

A Semantic Segmentation Method for Winter Wheat in North China Based on Improved HRNet

Winter wheat is one of the major crops for global food security. Accurate statistics of its planting area play a crucial role in agricultural policy formulation and resource management. However, the existing semantic segmentation methods for remote sensing images are subjected to limitations in dealing with noise, ambiguity, and intra-class heterogeneity, posing a negative impact on the segmentation performance of the spatial distribution and area of winter wheat fields in practical applications. In response to the above challenges, we proposed an improved HRNet-based semantic segmentation model in this paper. First, this model incorporates a semantic domain module (SDM), which improves the model’s precision of pixel-level semantic parsing and reduces the interference from noise through multi-confidence scale class representation. Second, a nested attention module (NAM) is embedded, which enhances the model’s capability of recognizing correct correlations in pixel classes. The experimental results show that the proposed model achieved a mean intersection over union (mIoU) of 80.51%, a precision of 88.64%, a recall of 89.14%, an overall accuracy (OA) of 90.12%, and an F1-score of 88.89% on the testing set. Compared to traditional methods, our model demonstrated better segmentation performance in winter wheat semantic segmentation tasks. The achievements of this study not only provide an effective tool and technical support for accurately measuring the area of winter wheat fields, but also have important practical value and profound strategic significance for optimizing agricultural resource allocation and achieving precision agriculture.

An One-step Triple Enhanced weakly supervised semantic segmentation using image-level labels.

Weakly supervised semantic segmentation, based on image-level labels, abandons the pixel-level labels relied upon by traditional semantic segmentation algorithms. It only utilizes images as supervision information, thereby reducing the time cost and human resources required for marking pixel data. The prevailing approach in weakly supervised segmentation involves two-step method, introducing an additional network and numerous parameters, thereby complicating the model structure. Furthermore, image-level labels typically furnishes only category information for the entire image, lacking specific location details and accurate target boundaries during model training. We propose an innovative One-Step Triple Enhanced weakly supervised semantic segmentation network(OSTE). OSTE streamlines the model structure, which can accomplish both pseudo-labels generation and semantic segmentation tasks in just one step. Furthermore, we augment the weakly supervised semantic segmentation network in three key aspects based on the class activation map construction method, thereby enhancing segmentation accuracy: Firstly, by integrating local information from the activation map with the image, we can enhance the network's localization and expansion capabilities to obtain more accurate and rich location information. Then, we refine the seed areas of the class activation map by exploiting the correlation between multi-level feature. Finally, we incorporate conditional random field theory to generate pseudo-labels with higher confidence and richer boundary information. In comparison to the prevailing two-step weakly supervised semantic segmentation schemes, the segmentation network proposed in this paper achieves a more competitive mean Intersection over Union (mIoU) score of 58.47% on Pascal VOC. Additionally, it enhances the mIoU score by at least 5.03% when compared to existing end-to-end schemes.

SwinUNeCCt: bidirectional hash-based agent transformer for cervical cancer MRI image multi-task learning

Cervical cancer is the fourth most common malignant tumor among women globally, posing a significant threat to women’s health. In 2022, approximately 600,000 new cases were reported, and 340,000 deaths occurred due to cervical cancer. Magnetic resonance imaging (MRI) is the preferred imaging method for diagnosing, staging, and evaluating cervical cancer. However, manual segmentation of MRI images is time-consuming and subjective. Therefore, there is an urgent need for automatic segmentation models to identify cervical cancer lesions in MRI scans accurately. All MRIs in our research are from cervical cancer patients diagnosed by pathology at Tongren City People’s Hospital. Strict data selection criteria and clearly defined inclusion and exclusion conditions were established to ensure data consistency and accuracy of research results. The dataset contains imaging data from 122 cervical cancer patients, with each patient having 100 pelvic dynamic contrast-enhanced MRI scans. Annotations were jointly completed by medical professionals from Universiti Putra Malaysia and the Radiology Department of Tongren City People’s Hospital to ensure data accuracy and reliability. Additionally, a novel computer-aided diagnosis model named SwinUNeCCt is proposed. This model incorporates (i) A bidirectional hash-based agent multi-head self-attention mechanism, which optimizes the interaction between local and global features in MRI, aiding in more accurate lesion identification. (ii) Reduced computational complexity of the self-attention mechanism. The effectiveness of the SwinUNeCCt model has been validated through comparisons with state-of-the-art 3D medical models, including nnUnet, TransBTS, nnFormer, UnetR, UnesT, SwinUNetR, and SwinUNeLCsT. In semantic segmentation tasks without a classification module, the SwinUNeCCt model demonstrates excellent performance across multiple key metrics: achieving a 95HD of 6.25, an IoU of 0.669, and a DSC of 0.802, all of which are the best results among the compared models. Simultaneously, SwinUNeCCt strikes a good balance between computational efficiency and model complexity, requiring only 442.7 GFLOPs of computational power and 71.2 M parameters. Furthermore, in semantic segmentation tasks that include a classification module, the SwinUNeCCt model also exhibits powerful recognition capabilities. Although this slightly increases computational overhead and model complexity, its performance surpasses other comparative models. The SwinUNeCCt model demonstrates excellent performance in semantic segmentation tasks, achieving the best results among state-of-the-art 3D medical models across multiple key metrics. It balances computational efficiency and model complexity well, maintaining high performance even with the inclusion of a classification module.

EIDU-Net: edge-preserved inception DenseGCN U-Net for LiDAR point cloud segmentation

With the development of laser scanners and machine learning, point cloud semantic segmentation plays a significant role in autonomous driving, scene reconstruction, human-computer interaction, and other fields. In recent years, point cloud semantic segmentation based on deep learning has become one of the key research directions in point cloud processing. Due to the limited ability to exploit geometric details and contextual information in point clouds, most methods that adopt encoder-decoder architecture lose local structural information easily, especially detailed features, and extract features insufficiently. To address this issue, the edge-preserving inception DenseGCN U-Net (entitled as EIDU-Net) is proposed. EIDU-Net makes full use of the complementation between geometric details in the original point cloud and high-level features. The edge-preserved graph pooling (EGP) layer, the key module of the EIDU-Net, is designed to retain additional edge feature information from the original point cloud during pooling operations. Accordingly, the edge-preserved graph unpooling (EGU) layer can restore the feature graph more efficiently based on the additionally retained edge features. Extensive experiments demonstrate that our proposed EIDU-Net has remarkable improvements on semantic segmentation tasks under whatever S3DIS or Terracotta Warrior fragments. Our code is publicly available at https://github.com/caoxin918/EIDU-Net.

LSL-SS-Net: level set loss-guided semantic segmentation networks for landslide extraction

ABSTRACT A landslide inventory is essential for numerous applications, including disaster prevention and mitigation. The emergence of advanced satellite technologies and the proliferation of extensive satellite imagery have greatly enhanced the role of deep learning-based semantic segmentation networks in the development of comprehensive landslide inventories. In deep learning, especially in semantic segmentation tasks, the loss function is crucial to guide the training of neural networks. However, typical loss functions have several challenges and shortcomings, including difficulty in handling objects of different scales and neglecting spatial correlation. These drawbacks hinder semantic segmentation networks from solely concentrating on accurately predicting landslide categories with rich semantic information. The spatial details (e.g. small landslides and precise landslide boundary information) in the segmentation results may be lost, reducing the accuracy of landslide extraction. Most studies only focus on training and detection in the local region. Consequently, detecting new unexplored region and detrimental to emergency response is challenging. This study presents a level set loss-guided semantic segmentation network, which can be integrated with different semantic segmentation networks, to address these issues. Three out of five study areas (e.g. Bijie, Jiuzhaigou, and Taitung) are used for model training and accuracy evaluation, while the rest, which are newly unexplored regions (e.g. Hokkaido and Haiti), are used to test the transferability of the proposed loss function in different regions. The study results validate the effectiveness of the designed level set loss in various semantic segmentation networks and in different study areas for landslide extraction.

ESFuse: Weak Edge Structure Perception Network for Infrared and Visible Image Fusion

Infrared and visible image fusion (IVIF) fully integrates the complementary features of different modal images, and the fused image provides a more comprehensive and objective interpretation of the scene compared to each source image, thus attracting extensive attention in the field of computer vision in recent years. However, current fusion methods usually center their attention on the extraction of prominent features, falling short of adequately safeguarding subtle and diminutive structures. To address this problem, we propose an end-to-end unsupervised IVIF method (ESFuse), which effectively enhances fine edges and small structures. In particular, we introduce a two-branch head interpreter to extract features from source images of different modalities. Subsequently, these features are fed into the edge refinement module with the detail injection module (DIM) to obtain the edge detection results of the source image, improving the network’s ability to capture and retain complex details as well as global information. Finally, we implemented a multiscale feature reconstruction module to obtain the final fusion results by combining the output of the DIM with the output of the head interpreter. Extensive IVIF fusion experiments on existing publicly available datasets show that the proposed ESFuse outperforms the state-of-the-art(SOTA) methods in both subjective vision and objective evaluation, and our fusion results perform well in semantic segmentation, target detection, pose estimation and depth estimation tasks. The source code has been availabled.

Enhancing bridge damage detection with Mamba-Enhanced HRNet for semantic segmentation.

With the acceleration of urbanization, bridges, as crucial infrastructure, their structural health and stability are paramount to public safety. This paper proposes Mamba-Enhanced HRNet for bridge damage detection. Mamba-Enhanced HRNet integrates the advantages of HRNet's multi-resolution parallel design and VMamba's visual state space model. By replacing the residual convolutional blocks in HRNet with a combination of VSS blocks and convolution, this model enhances the network's capability to capture global contextual information while maintaining computational efficiency. This work builds an extensive dataset with multiple damage kinds and uses Mean Intersection over Union (Mean IoU) as the assessment metric to assess the performance of Mamba-Enhanced HRNet. Experimental results demonstrate that Mamba-Enhanced HRNet achieves significant performance improvements in bridge damage semantic segmentation tasks, with Mean IoU scores of 0.963, outperforming several other semantic segmentation models.

An Object-Aware Network Embedding Deep Superpixel for Semantic Segmentation of Remote Sensing Images

Semantic segmentation forms the foundation for understanding very high resolution (VHR) remote sensing images, with extensive demand and practical application value. The convolutional neural networks (CNNs), known for their prowess in hierarchical feature representation, have dominated the field of semantic image segmentation. Recently, hierarchical vision transformers such as Swin have also shown excellent performance for semantic segmentation tasks. However, the hierarchical structure enlarges the receptive field to accumulate features and inevitably leads to the blurring of object boundaries. We introduce a novel object-aware network, Embedding deep SuperPixel, for VHR image semantic segmentation called ESPNet, which integrates advanced ConvNeXt and the learnable superpixel algorithm. Specifically, the developed task-oriented superpixel generation module can refine the results of the semantic segmentation branch by preserving object boundaries. This study reveals the capability of utilizing deep convolutional neural networks to accomplish both superpixel generation and semantic segmentation of VHR images within an integrated end-to-end framework. The proposed method achieved mIoU scores of 84.32, 90.13, and 55.73 on the Vaihingen, Potsdam, and LoveDA datasets, respectively. These results indicate that our model surpasses the current advanced methods, thus demonstrating the effectiveness of the proposed scheme.

Unsupervised Multi-Scale Hybrid Feature Extraction Network for Semantic Segmentation of High-Resolution Remote Sensing Images

Generating pixel-level annotations for semantic segmentation tasks of high-resolution remote sensing images is both time-consuming and labor-intensive, which has led to increased interest in unsupervised methods. Therefore, in this paper, we propose an unsupervised multi-scale hybrid feature extraction network based on the CNN-Transformer architecture, referred to as MSHFE-Net. The MSHFE-Net consists of three main modules: a Multi-Scale Pixel-Guided CNN Encoder, a Multi-Scale Aggregation Transformer Encoder, and a Parallel Attention Fusion Module. The Multi-Scale Pixel-Guided CNN Encoder is designed for multi-scale, fine-grained feature extraction in unsupervised tasks, efficiently recovering local spatial information in images. Meanwhile, the Multi-Scale Aggregation Transformer Encoder introduces a multi-scale aggregation module, which further enhances the unsupervised acquisition of multi-scale contextual information, obtaining global features with stronger feature representation. The Parallel Attention Fusion Module employs an attention mechanism to fuse global and local features in both channel and spatial dimensions in parallel, enriching the semantic relations extracted during unsupervised training and improving the performance of unsupervised semantic segmentation. K-means clustering is then performed on the fused features to achieve high-precision unsupervised semantic segmentation. Experiments with MSHFE-Net on the Potsdam and Vaihingen datasets demonstrate its effectiveness in significantly improving the accuracy of unsupervised semantic segmentation.

A semantic segmentation method integrated convolutional nonlinear spiking neural model with Transformer

Semantic segmentation is a critical task in computer vision, with significant applications in areas like autonomous driving and medical imaging. Transformer-based methods have gained considerable attention recently because of their strength in capturing global information. However, these methods often sacrifice detailed information due to the lack of mechanisms for local interactions. Similarly, convolutional neural network (CNN) methods struggle to capture global context due to the inherent limitations of convolutional kernels. To overcome these challenges, this paper introduces a novel Transformer-based semantic segmentation method called NSNPFormer, which leverages the nonlinear spiking neural P (NSNP) system—a computational model inspired by the spiking mechanisms of biological neurons. The NSNPFormer employs an encoding–decoding structure with two convolutional NSNP components and a residual connection channel. The convolutional NSNP components facilitate nonlinear local feature extraction and block-level feature fusion. Meanwhile, the residual connection channel helps prevent the loss of feature information during the decoding process. Evaluations on the ADE20K and Pascal Context datasets show that NSNPFormer achieves mIoU scores of 53.7 and 58.06, respectively, highlighting its effectiveness in semantic segmentation tasks.

Pixel shuffling is all you need: spatially aware convmixer for dense prediction tasks

ConvMixer is an extremely simple model that could perform better than the state-of-the-art convolutional-based and vision transformer-based methods thanks to mixing the input image patches using a standard convolution. The global mixing process of the patches is only valid for the classification tasks, but it cannot be used for dense prediction tasks as the spatial information of the image is lost in the mixing process. We propose a more efficient technique for image patching, known as pixel shuffling, as it can preserve spatial information. We downsample the input image using the pixel shuffle downsampling in the same form of image patches so that the ConvMixer can be extended for the dense prediction tasks. This paper proves that pixel shuffle downsampling is more efficient than the standard image patching as it outperforms the original ConvMixer architecture in the CIFAR10 and ImageNet-1k classification tasks. We also suggest spatially-aware ConvMixer architectures based on efficient pixel shuffle downsampling and upsampling operations for semantic segmentation and monocular depth estimation. We performed extensive experiments to test the proposed architectures on several datasets; Pascal VOC2012, Cityscapes, and ADE20k for semantic segmentation, NYU-depthV2, and Cityscapes for depth estimation. We show that SA-ConvMixer is efficient enough to get relatively high accuracy at many tasks in a few training epochs (150∼400). The proposed SA-ConvMixer could achieve an ImageNet-1K Top-1 classification accuracy of 87.02%, mean intersection over union (mIOU) of 87.1% in the PASCAL VOC2012 semantic segmentation task, and absolute relative error of 0.096 in the NYU depthv2 depth estimation task. The implementation code of the proposed method is available at: https://github.com/HatemHosam/SA-ConvMixer/.

Accelerating Neural Network Inference in Handwritten Digit Recognition — Comparative Study

In deep learning, one of the popular subclasses of multilayer perceptron is the Convolutional Neural Networks (CNNs), optimized for image and semantic segmentation tasks. With a large number of floating-point operations and relatively less data transfer in the training phase, CNNs are well suited to be handled by parallel architectures. The high accuracy of CNNs comes at the cost of consequential compute and memory demands. In this article, we present a comparative analysis of a pre-trained CNN framework for recognizing a handwritten digit on different processing platforms. The performance of the neural network as the function of the parallel processing platforms offers parametric insights and factors that influence the inference on state-of-the-art Graphics Processing Units (GPUs) and systolic arrays such as Eyeriss and Tensor Processing Unit (TPU). Through inference time analysis, we observed that the systolic arrays can outperform upto 58.7 times better than a Turing architecture powered GPU. We show that while the efficiency with which the available resources are utilized is higher in the existing GPUs (upto 32%) than the TPUs, the efficiency of application-specific systolic arrays can be on par with that of GPUs. We present the results obtained from three different customized systolic array-based platforms which can be adopted by the designers to decide the hardware optimization goal.

SFINet: A semantic feature interactive learning network for full-time infrared and visible image fusion

Infrared and visible image fusion aims to combine data from various source images to generate a high-quality image. Nevertheless, numerous fusion methods often prioritize visual quality above semantic information. To address this problem, we present a Semantic Feature Interactive Learning Network (SFINet) for full-time infrared and visible images. The SFINet encompasses an image fusion network and an image segmentation network through a Semantic Feature Interaction (SFI) module. The image fusion network employs Multi-scale Feature Extraction (MFE) modules to capture global and local information at multiple scales. Meanwhile, it performs an adaptive fusion of complementary information using a Dual Attention Feature Fusion (DAFF) module. The image segmentation network guides the image fusion network using the SFI module for semantic feature interaction. Comparative results prove that the proposed method is superior to state-of-the-art (SOTA) models in image fusion and semantic segmentation tasks. The code is available at https://github.com/songwenhao123/SFINet.

Found missing semantics: Supplemental prototype network for few-shot semantic segmentation

Few-shot semantic segmentation alleviates the problem of massive data requirements and high costs in semantic segmentation tasks. By learning from support set, few-shot semantic segmentation can segment new classes. However, existing few-shot semantic segmentation methods suffer from information loss during the process of mask average pooling. To address this problem, we propose a supplemental prototype network (SPNet). The SPNet aggregates the lost information from global prototypes to create a supplemental prototype, which enhances the segmentation performance for the current class. In addition, we utilize mutual attention to enhance the similarity between the support and the query feature maps, allowing the model to better identify the target to be segmented. Finally, we introduce a Self-correcting auxiliary, which utilizes the data more effectively to improve segmentation accuracy. We conducted extensive experiments on PASCAL-5i and COCO-20i, which demonstrated the effectiveness of SPNet. And our method achieved state-of-the-art results in the 1-shot and 5-shot semantic segmentation settings.

Leveraging CAM Algorithms for Explaining Medical Semantic Segmentation

Convolutional neural networks (CNNs) achieve prevailing results in segmentation tasks nowadays and represent the state-of-the-art for image-based analysis. However, the under- standing of the accurate decision-making process of a CNN is rather unknown. The research area of explainable artificial intelligence (xAI) primarily revolves around understanding and interpreting this black-box behavior. One way of interpreting a CNN is the use of class ac- tivation maps (CAMs) that represent heatmaps to indicate the importance of image areas for the prediction of the CNN. For classification tasks, a variety of CAM algorithms exist. But for segmentation tasks, only one CAM algorithm for the interpretation of the output of a CNN exist. We propose a transfer between existing classification- and segmentation- based methods for more detailed, explainable, and consistent results which show salient pixels in semantic segmentation tasks. The resulting Seg-HiRes-Grad CAM is an exten- sion of the segmentation-based Seg-Grad CAM with the transfer to the classification-based HiRes CAM. Our method improves the previously-mentioned existing segmentation-based method by adjusting it to recently published classification-based methods. Especially for medical image segmentation, this transfer solves existing explainability disadvantages. The code is available at <a href='https://github.com/TillmannRheude/SegHiResGrad_CAM'>https://github.com/TillmannRheude/SegHiResGrad_CAM</a>

Multi-instance imbalance semantic segmentation by instance-dependent attention and adaptive hard instance mining

Multi-instance imbalanced semantic segmentation (MISS) is a common issue where larger foreground instances can dominate smaller ones, yet still produce satisfactory Dice. Detecting small instances is crucial for many applications. For example, in the diagnosis of coronary artery disease, it is essential to locate small-scale lesions. To address the challenge of MISS, we propose MISS-Net (MISS-Network), which considers the distribution characteristics and segmentation difficulty of instances. On one hand, given the spatially sparse distribution characteristics of instances, we propose an instance-dependent SparseViT. SparseViT learns unstructured sparse attention to capture instance-related patterns by computing content-aware attention scores and selecting the top-K indices. On the other hand, we propose a hard instance-aware loss that identifies ‘hard-to-segment’ instances through a dynamic instance weighting strategy. Combined with a global segmentation loss, it produces a trade-off between the global-level and instance-level segmentation performance. We extensively evaluated MISS-Net on three challenging 3D semantic segmentation tasks: two coronary artery calcification (CAC) datasets and one liver tumor (LITS) dataset. These datasets contain a large number of instances with significant variability in instance volumes (e.g., in the LITS dataset, a patient has an average of 12.39 instances, with instance volumes ranging from 987.66cm3 to 0.04cm3). The results demonstrate that MISS-Net significantly improves both instance-level and global-level segmentation performance. For the CAC task, MISS-Net improves Dice by 3.6% and 6.4%, and instance F1 by 3.1% and 1.9%. For the LITS dataset, Dice and instance F1 are improved by 1.6% and 3.8%, respectively.

Real-Time Semantic Segmentation Algorithm for Street Scenes Based on Attention Mechanism and Feature Fusion

In computer vision, the task of semantic segmentation is crucial for applications such as autonomous driving and intelligent surveillance. However, achieving a balance between real-time performance and segmentation accuracy remains a significant challenge. Although Fast-SCNN is favored for its efficiency and low computational complexity, it still faces difficulties when handling complex street scene images. To address this issue, this paper presents an improved Fast-SCNN, aiming to enhance the accuracy and efficiency of semantic segmentation by incorporating a novel attention mechanism and an enhanced feature extraction module. Firstly, the integrated SimAM (Simple, Parameter-Free Attention Module) increases the network’s sensitivity to critical regions of the image and effectively adjusts the feature space weights across channels. Additionally, the refined pyramid pooling module in the global feature extraction module captures a broader range of contextual information through refined pooling levels. During the feature fusion stage, the introduction of an enhanced DAB (Depthwise Asymmetric Bottleneck) block and SE (Squeeze-and-Excitation) attention optimizes the network’s ability to process multi-scale information. Furthermore, the classifier module is extended by incorporating deeper convolutions and more complex convolutional structures, leading to a further improvement in model performance. These enhancements significantly improve the model’s ability to capture details and overall segmentation performance. Experimental results demonstrate that the proposed method excels in processing complex street scene images, achieving a mean Intersection over Union (mIoU) of 71.7% and 69.4% on the Cityscapes and CamVid datasets, respectively, while maintaining inference speeds of 81.4 fps and 113.6 fps. These results indicate that the proposed model effectively improves segmentation quality in complex street scenes while ensuring real-time processing capabilities.