Scene Parsing Research Articles

A progressive segmentation network for navigable areas with semantic–spatial information flow

Segmentation of safe navigable areas is a crucial technology for scene parsing in autopilot systems. However, existing segmentation methods often fail to adequately exploit the complementary relationships between multiscale features in complex wild environments, resulting in insufficient information fusion. In pursuit of a resolution, a Progressive Segmentation Network (PSNet) is proposed for navigable areas segmentation, which builds a semantic–spatial information flow branch to dynamically exploit the complementary relationships between multiscale features for progressive guided learning. Specifically, PSNet consists of four essential modules, including the Local Capturer and Global dependence Builder (LCGB), Multi-Directional Pooling Module (MDPM), Fusion-wise Module (FWM), and Spatial Weight Aggregation Module (SWAM). To facilitate efficient information dissemination, the LCGB was utilized to capture dense spatial information, and the MDPM was proposed to extract global geometric information of obstacles. These pieces of information serve as prior knowledge to guide learning. Additionally, we propose the FWM based on attention fusion unit (AFU) and contribution weights unit (CWU) to construct the complementary relationships between multiscale features and obtain rich multiscale fusion information. SWAM is proposed to enhance the significant spatial features from FWM and achieve impressive segmentation results. Extensive experimental results on the wild datasets demonstrate that PSNet outperforms state-of-the-art methods in recognizing safe navigable areas. The code for PSNet will be available at https://github.com/lv881314/PSNet.

YOLO-SegNet: A Method for Individual Street Tree Segmentation Based on the Improved YOLOv8 and the SegFormer Network

In urban forest management, individual street tree segmentation is a fundamental method to obtain tree phenotypes, which is especially critical. Most existing tree image segmentation models have been evaluated on smaller datasets and lack experimental verification on larger, publicly available datasets. Therefore, this paper, based on a large, publicly available urban street tree dataset, proposes YOLO-SegNet for individual street tree segmentation. In the first stage of the street tree object detection task, the BiFormer attention mechanism was introduced into the YOLOv8 network to increase the contextual information extraction and improve the ability of the network to detect multiscale and multishaped targets. In the second-stage street tree segmentation task, the SegFormer network was proposed to obtain street tree edge information more efficiently. The experimental results indicate that our proposed YOLO-SegNet method, which combines YOLOv8+BiFormer and SegFormer, achieved a 92.0% mean intersection over union (mIoU), 95.9% mean pixel accuracy (mPA), and 97.4% accuracy on a large, publicly available urban street tree dataset. Compared with those of the fully convolutional neural network (FCN), lite-reduced atrous spatial pyramid pooling (LR-ASPP), pyramid scene parsing network (PSPNet), UNet, DeepLabv3+, and HRNet, the mIoUs of our YOLO-SegNet increased by 10.5, 9.7, 5.0, 6.8, 4.5, and 2.7 percentage points, respectively. The proposed method can effectively support smart agroforestry development.

High-resolution mapping of grassland canopy cover in China through the integration of extensive drone imagery and satellite data

Canopy cover is a crucial indicator for assessing grassland health and ecosystem services. However, achieving accurate high-resolution estimates of grassland canopy cover at a large spatial scale remains challenging due to the limited spatial coverage of field measurements and the scale mismatch between field measurements and satellite imagery. In this study, we addressed these challenges by proposing a regression-based approach to estimate large-scale grassland canopy cover, leveraging the integration of drone imagery and multisource remote sensing data. Specifically, over 90,000 10 × 10 m drone image tiles were collected at 1,255 sites across China. All drone image tiles were classified into grass and non-grass pixels to generate ground-truth canopy cover estimates. These estimates were then temporally aligned with satellite imagery-derived features to build a random forest regression model to map the grassland canopy cover distribution of China. Our results revealed that a single classification model can effectively distinguish between grass and non-grass pixels in drone images collected across diverse grassland types and large spatial scales, with multilayer perceptron demonstrating superior classification accuracy compared to Canopeo, support vector machine, random forest, and pyramid scene parsing network. The integration of extensive drone imagery successfully addressed the scale-mismatch issue between traditional ground measurements and satellite imagery, contributing significantly to enhancing mapping accuracy. The national canopy cover map of China generated for the year 2021 exhibited a spatial pattern of increasing canopy cover from northwest to southeast, with an average value of 56 % and a standard deviation of 26 %. Moreover, it demonstrated high accuracy, with a coefficient of determination of 0.89 and a root-mean-squared error of 12.38 %. The resulting high-resolution canopy cover map of China holds great potential in advancing our comprehension of grassland ecosystem processes and advocating for the sustainable management of grassland resources.

Refined Intelligent Landslide Identification Based on Multi-Source Information Fusion

Landslides are most severe in the mountainous regions of southwestern China. While landslide identification provides a foundation for disaster prevention operations, methods for utilizing multi-source data and deep learning techniques to improve the efficiency and accuracy of landslide identification in complex environments are still a focus of research and a difficult issue in landslide research. In this study, we address the above problems and construct a landslide identification model based on the shifted window (Swin) transformer. We chose Ya’an, which has a complex terrain and experiences frequent landslides, as the study area. Our model, which fuses features from different remote sensing data sources and introduces a loss function that better learns the boundary information of the target, is compared with the pyramid scene parsing network (PSPNet), the unified perception parsing network (UPerNet), and DeepLab_V3+ models in order to explore the learning potential of the model and test the models’ resilience in an open-source landslide database. The results show that in the Ya’an landslide database, compared with the above benchmark networks (UPerNet, PSPNet, and DeepLab_v3+), the Swin Transformer-based optimization model improves overall accuracies by 1.7%, 2.1%, and 1.5%, respectively; the F1_score is improved by 14.5%, 16.2%, and 12.4%; and the intersection over union (IoU) is improved by 16.9%, 18.5%, and 14.6%, respectively. The performance of the optimized model is excellent.

HEFANet: hierarchical efficient fusion and aggregation segmentation network for enhanced rgb-thermal urban scene parsing

HEFANet: hierarchical efficient fusion and aggregation segmentation network for enhanced rgb-thermal urban scene parsing

Hybrid learning architecture for high-speed railroad scene parsing and potential safety hazard evaluation of UAV images

Potential safety hazards (PSHs) around tracks need to be accurately and timely detected to ensure high- speed railroad operating safety. Unmanned aerial vehicle (UAV)-based railroad inspection has potential to complement visual inspections by providing better overhead views and mitigating safety concerns. This paper proposes a hybrid learning architecture, named YOLORS (you only look once railroad scene), to parse driveways and tracks in UAV images, and evaluate PSHs along tracks. First, a stripe-pool-based parsing branch is designed and assembled in YOLORS to precisely accomplish the pixel-level parsing tasks. Second, a new data augmentation method called object-guided mosaic is developed and incorporated into the architecture for improving small object detection performance. Finally, extensive experiments indicate YOLORS demonstrates exceptional performance in both pixel-level parsing and object detection tasks, which outperforms the current advanced models. This proposed approach can quickly convert UAV imagery containing complicated railroad scenes into useful information with a high detection rate.

CLIP-SP: Vision-language model with adaptive prompting for scene parsing

We present a novel framework, CLIP-SP, and a novel adaptive prompt method to leverage pre-trained knowledge from CLIP for scene parsing. Our approach addresses the limitations of DenseCLIP, which demonstrates the superior image segmentation provided by CLIP pre-trained models over ImageNet pre-trained models, but struggles with rough pixel-text score maps for complex scene parsing. We argue that, as they contain all textual information in a dataset, the pixel-text score maps, i.e., dense prompts, are inevitably mixed with noise. To overcome this challenge, we propose a two-step method. Firstly, we extract visual and language features and perform multi-label classification to identify the most likely categories in the input images. Secondly, based on the top-k categories and confidence scores, our method generates scene tokens which can be treated as adaptive prompts for implicit modeling of scenes, and incorporates them into the visual features fed into the decoder for segmentation. Our method imposes a constraint on prompts and suppresses the probability of irrelevant categories appearing in the scene parsing results. Our method achieves competitive performance, limited by the available visual-language pre-trained models. Our CLIP-SP performs 1.14% better (in terms of mIoU) than DenseCLIP on ADE20K, using a ResNet-50 backbone.

Quantifying Geographic Atrophy in Age-Related Macular Degeneration: A Comparative Analysis Across 12 Deep Learning Models.

AI algorithms have shown impressive performance in segmenting geographic atrophy (GA) from fundus autofluorescence (FAF) images. However, selection of artificial intelligence (AI) architecture is an important variable in model development. Here, we explore 12 distinct AI architecture combinations to determine the most effective approach for GA segmentation. We investigated various AI architectures, each with distinct combinations of encoders and decoders. The architectures included three decoders-FPN (Feature Pyramid Network), UNet, and PSPNet (Pyramid Scene Parsing Network)-and serve as the foundation framework for segmentation task. Encoders including EfficientNet, ResNet (Residual Networks), VGG (Visual Geometry Group) and Mix Vision Transformer (mViT) have a role in extracting optimum latent features for accurate GA segmentation. Performance was measured through comparison of GA areas between human and AI predictions and Dice Coefficient (DC). The training dataset included 601 FAF images from AREDS2 study and validation included 156 FAF images from the GlaxoSmithKline study. The mean absolute difference between grader measured and AI predicted areas ranged from -0.08 (95% CI = -1.35, 1.19) to 0.73 mm2 (95% CI = -5.75,4.29) and DC between 0.884-0.993. The best-performing models were UNet and FPN frameworks with mViT, and the least-performing models were PSPNet framework. The choice of AI architecture impacts GA segmentation performance. Vision transformers with FPN and UNet architectures demonstrate stronger suitability for this task compared to Convolutional Neural Network- and PSPNet-based models. Selecting an AI architecture must be tailored to the specific goals of the project, and developers should consider which architecture is ideal for their project.

Feature boosting with efficient attention for scene parsing

The complexity of scene parsing grows with the number of object and scene classes, which is higher in unrestricted open scenes. The biggest challenge is to model the spatial relation between scene elements while succeeding in identifying objects at smaller scales. This paper presents a novel feature-boosting network that gathers spatial context from multiple levels of feature extraction and computes the attention weights for each level of representation to generate the final class labels. A novel ‘channel attention module’ is designed to compute the attention weights, ensuring that features from the relevant extraction stages are boosted while the others are attenuated. The model also learns spatial context information at low resolution to preserve the abstract spatial relationships among scene elements and reduce computational cost. Spatial attention is subsequently concatenated into a final feature set before applying feature boosting. Low-resolution spatial attention features are trained using an auxiliary task that help to learn a coarse global scene structure. The proposed model outperforms all state-of-the-art models on both the ADE20K and the Cityscapes datasets.

RoadFormer: Duplex Transformer for RGB-Normal Semantic Road Scene Parsing

RoadFormer: Duplex Transformer for RGB-Normal Semantic Road Scene Parsing

Automated structural repair based on continuous carbon fiber reinforced plastic 3D printing and online model reconstruction

AbstractAdditive manufacturing of continuous carbon fiber reinforced plastic (CCFRP) has gained increasing prominence in structural repair through material accumulation. However, the repairing process entails laborious preliminary tasks, such as positioning, model reconstruction, and data input. In this article, computer vision is integrated into a CCFRP 3D printing device and a printing path‐selecting strategy is proposed to automate the repair process with accuracy and efficiency. An irregularly damaged structure is placed arbitrarily on the printing platform. Subsequently, the damaged area is extracted by semantic segmentation to construct a 3D model. The printing paths including zig‐zag and contour offset infilling for an irregular damage area are selected, according to the prediction of flexural modulus based on the finite element method. Finally, an in‐situ composite printing is conducted based on coaxial extrusion principle. The results show that the intersection over union (IOU) of regular damaged areas by the applied pyramid scene parsing network (PSPNet) segmentation model is 0.81. Semantic segmentation is more robust than conventional feature extraction based on filtering methods. The appearance of the repaired structure is consistent with that of the surrounding structure. The combination of CV and additive manufacturing technology presents a new way to simplify the repair process and improve repair effects.Highlights An automated repairing system for irregular damaged areas is established. Deep learning for semantic segmentation is integrated into 3D printing. Finite element submodule is introduced to select optimal printing paths. The CCFRP 3D printing is introduced to surface defect repairing.

Crop Land Change Detection with MC&N-PSPNet

To enhance the accuracy of agricultural area classification and enable remote sensing monitoring of agricultural regions, this paper investigates classification models and their application in change detection within rural areas, proposing the MC&N-PSPNet (CBAM into MobileNetV2 and NAM into PSPNet) network model. Initially, the HRSCD (High Resolution Semantic Change Detection) dataset labels undergo binary redrawing. Subsequently, to efficiently extract image features, the original PSPNet (Pyramid Scene Parsing Network) backbone network, ResNet50 (Residual Network-50), is substituted with the MobileNetV2 (Inverted Residuals and Linear Bottlenecks) model. Furthermore, to enhance the model’s training efficiency and classification accuracy, the NAM (Normalization-Based Attention Module) attention mechanism is introduced into the improved PSPNet model to obtain the categories of land cover changes in remote sensing images before and after the designated periods. Finally, the final change detection results are obtained by performing a different operation on the classification results for different periods. Through experimental analysis, this paper demonstrates the proposed method’s superior capability in segmenting agricultural areas, which is crucial for effective agricultural area change detection. The model achieves commendable performance metrics, including overall accuracy, Kappa value, MIoU, and MPA values of 95.03%, 88.15%, 93.55%, and 88.90%, respectively, surpassing other models. Moreover, the model exhibits robust performance in final change detection, achieving an overall accuracy and Kappa value of 93.24% and 92.29%, respectively. The results of this study show that the MC&N-PSPNet model has significant advantages in the detection of changes in agricultural zones, which provides a scientific basis and technical support for agricultural resource management and policy formulation.

NDNet: Spacewise Multiscale Representation Learning via Neighbor Decoupling for Real-Time Driving Scene Parsing.

As a safety-critical application, autonomous driving requires high-quality semantic segmentation and real-time performance for deployment. Existing method commonly suffers from information loss and massive computational burden due to high-resolution input-output and multiscale learning scheme, which runs counter to the real-time requirements. In contrast to channelwise information modeling commonly adopted by modern networks, in this article, we propose a novel real-time driving scene parsing framework named NDNet from a novel perspective of spacewise neighbor decoupling (ND) and neighbor coupling (NC). We first define and implement the reversible operations called ND and NC, which realize lossless resolution conversion for complementary thumbnails sampling and collation to facilitate spatial modeling. Based on ND and NC, we further propose three modules, namely, local capturer and global dependence builder (LCGB), spacewise multiscale feature extractor (SMFE), and high-resolution semantic generator (HSG), which form the whole pipeline of NDNet. The LCGB serves as a stem block to preprocess the large-scale input for fast but lossless resolution reduction and extract initial features with global context. Then the SMFE is used for dense feature extraction and can obtain rich multiscale features in spatial dimension with less computational overhead. As for high-resolution semantic output, the HSG is designed for fast resolution reconstruction and adaptive semantic confusion amending. Experiments show the superiority of the proposed method. NDNet achieves the state-of-the-art performance on the Cityscapes dataset which reports 76.47% mIoU at 240+ frames/s and 78.8% mIoU at 150+ frames/s on the benchmark. Codes are available at https://github.com/LiShuTJ/NDNet.

Research on Improved Road Visual Navigation Recognition Method Based on DeepLabV3+ in Pitaya Orchard

Traditional DeepLabV3+ image semantic segmentation methods face challenges in pitaya orchard environments characterized by multiple interference factors, complex image backgrounds, high computational complexity, and extensive memory consumption. This paper introduces an improved visual navigation path recognition method for pitaya orchards. Initially, DeepLabV3+ utilizes a lightweight MobileNetV2 as its primary feature extraction backbone, which is augmented with a Pyramid Split Attention (PSA) module placed after the Atrous Spatial Pyramid Pooling (ASPP) module. This improvement enhances the spatial feature representation of feature maps, thereby sharpening the segmentation boundaries. Additionally, an Efficient Channel Attention Network (ECANet) mechanism is integrated with the lower-level features of MobileNetV2 to reduce computational complexity and refine the clarity of target boundaries. The paper also designs a navigation path extraction algorithm, which fits the road mask regions segmented by the model to achieve precise navigation path recognition. Experimental findings show that the enhanced DeepLabV3+ model achieved a Mean Intersection over Union (MIoU) and average pixel accuracy of 95.79% and 97.81%, respectively. These figures represent increases of 0.59 and 0.41 percentage points when contrasted with the original model. Furthermore, the model’s memory consumption is reduced by 85.64%, 84.70%, and 85.06% when contrasted with the Pyramid Scene Parsing Network (PSPNet), U-Net, and Fully Convolutional Network (FCN) models, respectively. This reduction makes the proposed model more efficient while maintaining high segmentation accuracy, thus supporting enhanced operational efficiency in practical applications. The test results for navigation path recognition accuracy reveal that the angle error between the navigation centerline extracted using the least squares method and the manually fitted centerline is less than 5°. Additionally, the average deviation between the road centerlines extracted under three different lighting conditions and the actual road centerline is only 2.66 pixels, with an average image recognition time of 0.10 s. This performance suggests that the study can provide an effective reference for visual navigation in smart agriculture.

Pixel-Level Crack Identification for Bridge Concrete Structures Using Unmanned Aerial Vehicle Photography and Deep Learning

Traditional manual inspection technology has the problems of high risk, low efficiency, and being time-consuming in bridge safety management. The unmanned aerial vehicle (UAV)-based detection technology is widely used in bridge structure safety monitoring. However, the existing deep learning-based concrete crack identification method has great limitations in dealing with complex background and tiny cracks in bridge structures. To address these problems, this study designs a crack pixel-level high-performance segmentation model for bridge concrete cracks that is suitable for UAV detection scenarios using machine vision (MV) and deep learning (DL) algorithms. First, considering the high requirements for the computing performance of the MV-based model for UAV-based detection, the ResNet-18-based lightweight convolutional neural network is used to represent the traditional large-scale backbone network of the pyramid scene parsing network (PSPNet) to develop a high-performance crack automatic identification model. Then, considering that bridge concrete cracks have the characteristics of subtle shapes and complex backgrounds, the spatial position self-attention module is inserted into the PSPNet to improve its detection accuracy. A concrete bridge is used for the case study, and a dataset of cracks in bridge concrete structures collected by UAVs is constructed and used for model training. The experimental results show that the loss function of the developed method in the training process results in a smooth decline, and the developed algorithm achieves the evaluation indicators of 0.9008 precision, 0.8750 recall, 0.8820 accuracy, and 0.9012 IOU on the bridge concrete crack dataset, which are significantly higher than other state-of-the-art baseline methods. In addition, four common UAV bridge detection scenarios, including low light, complex crack forms, high background roughness, and complex background scenes, are used to further test the crack detection ability of the developed crack identification model. The experimental results show that the proposed crack identification method can effectively overcome interference and real-size pixel-level segmentation of crack morphology. In addition, it also achieved a detection efficiency of 35.04 FPS, which shows that the real-time detection ability of the method has good applicability in the UAV detection scene.

PSO-PSP-Net + InceptionV3: An optimized hyper-parameter tuned Computer-Aided Diagnostic model for liver tumor detection using CT scan slices

An automated diagnostic system leads to a supreme requirement in medical image analysis, greatly impacting the death rate due to the high spreading rate of liver tumors. However, the traditional histopathological diagnostic processes cause more disruption in the medical detection framework because of the procedure of manual estimation of CT scans and medical labor-intensive, which lead to the time consumption process. It is a highly challenging task for the segmentation of liver tumor because of the irregular appearance, distinct densities, patchy edges, and tumor sizes. To overcome this issue, an automated computer-aided diagnosis framework has been utilized to diagnose the liver cancer quickly and accurately with the less workload of expert radiologists.Presently, Deep learning-based techniques have been applied to identify the tumor regions and assist radiologists in the initial diagnosis and scrutinizing the severity level. The proposed work presents a modified pyramid scene parsing network (PSP-Net) model based on InceptionV3 deep network architecture with a particle swarm optimization algorithm, which increases the efficiency and effectiveness of the liver diagnostic model. Three publicly accessible benchmark datasets have been engaged with the pre-processing task accomplished by image normalization and data augmentation techniques. Moreover, a hyper-parameter tuned improved PSO-PSP-Net algorithm has analyzed the optimal set of gathered features. In the experimental evaluation, the proposed model outperformed than grey wolf optimization (GWO-PSP-Net), PSO-U-Net, GWO-U-Net, opposition based spotted hyena optimization (O-SHO-CRNN), and grey wolf-class topper optimization (GW-CTO) + U-Net by 8.42 %, 4.9 %, 9.54 %, 5.12 %, and 1.75 %, respectively in terms of accuracy for LiTS dataset. The developed PSO-PSP-Net model attained better accuracy than GWO-PSP-Net, PSO-U-Net, GWO-U-Net, and Residual Atrous U-Net (RA-U-Net) for the segmentation of Type-I and Type-II liver tumors by 6.88 %, 8.44 %, 13.75 %, 5.99 % and 6.12 %, 9.74 %, 17.3 %, and 5.11 %, respectively for 3DIRCAD-b1 dataset. The obtained accuracy of the developed model is 14.09 %, 7.54 %, 16.36 %, 2.64 %, and 14.9 %, 8.86 %, 17.76 %, and 4.87 % maximized than GWO-PSP-Net, PSO-U-Net, GWO-U-Net, and RA-U-Net, respectively, for the segmentation of the infected arteries and bones in the 3DIRCAD-b1 dataset. For the CHAOS dataset, the improved PSO-PSP-Net model outperformed GWO-PSP-Net, PSO-U-Net, and GWO-U-Net, respectively, in terms of overall accuracy by 7.44 %, 8.85 %, and 17.2 %, respectively. The investigational analysis validates the proposed PSO-PSP-Net model's effectiveness and performance in obtaining high segmentation accuracy in comparison with other state-of-the-art approaches.

Misalignment fusion network for parsing infrared and visible urban scenes

In the realm of infrared and visible scene parsing, satisfactory performance has been achieved by leveraging the complementary nature of infrared and visible imaging modalities. Existing methods have employed various strategies to fuse cross-modality features. However, these strategies typically integrate features at the same levels (the depths of network) neglecting the potential interactions across different levels. To address this limitation, we introduce a novel concept called misalignment fusion, which involves merging multimodality feature maps at distinct levels. In line with this concept, we propose a misalignment fusion network (MFNet) specifically designed for the task of infrared and visible urban scene parsing. Our network incorporates a misalignment-guided fusion module to integrate cross-modality features, as well as an adaptive refined selective fusion module to combine the predicted segmentation maps obtained from two parallel-branch decoders. Numerous experiments have been conducted to evaluate the performance of the proposed MFNet. The results consistently demonstrate that our approach surpasses existing state-of-the-art methods in the field of infrared and visible urban scene parsing.

Continuous Refinement-based Digital Pathology Image Assistance Scheme in Medical Decision-Making Systems.

Digital pathology images' extensive cellular information provide a trustworthy foundation for tumor diagnosis. With the aid of computer-aided diagnostics, pathologists can locate crucial information more quickly. The cascade structure refines the segmentation results by utilizing its multi-task and multi-stage characteristics. However, cascade-based models require downsampling and cropping of patches during the inference process due to the ultra-high resolution and complex structure of pathology images. This not only increases the cost and computation time but also results in the loss of cellular details and corrupts the global contextual information. This study proposes a Digital Pathology Image Assistance Program (CRSDPI) for medical decision-making systems that is based on continuous improvement. After locating the region of interest using the maximum inter-class variance method, the pictures are preprocessed to account for the impacts of staining inconsistencies and sensitivity variations on the model's performance. Ultimately, we create a two-phase continuously refined segmentation network (TCRNet) by combining an enhanced continuous refinement model with a coarse segmentation network built on a pyramid scene parsing network. The coarse segmentation network introduces an auxiliary loss term to speed up convergence, and the refined model introduces an implicit function to reduce computational cost and reconstruct more details. The TCRNet model refines the target by successively aligning the features without the need to take cascading decoder operations after encoder. Experiments conducted on digital pathology images of breast cancer and osteosarcoma demonstrate the superior prediction accuracy and computational speed of our strategy.

A transformer-based approach empowered by a self-attention technique for semantic segmentation in remote sensing

Semantic segmentation of Remote Sensing (RS) images involves the classification of each pixel in a satellite image into distinct and non-overlapping regions or segments. This task is crucial in various domains, including land cover classification, autonomous driving, and scene understanding. While deep learning has shown promising results, there is limited research that specifically addresses the challenge of processing fine details in RS images while also considering the high computational demands. To tackle this issue, we propose a novel approach that combines convolutional and transformer architectures. Our design incorporates convolutional layers with a low receptive field to generate fine-grained feature maps for small objects in very high-resolution images. On the other hand, transformer blocks are utilized to capture contextual information from the input. By leveraging convolution and self-attention in this manner, we reduce the need for extensive downsampling and enable the network to work with full-resolution features, which is particularly beneficial for handling small objects. Additionally, our approach eliminates the requirement for vast datasets, which is often necessary for purely transformer-based networks. In our experimental results, we demonstrate the effectiveness of our method in generating local and contextual features using convolutional and transformer layers, respectively. Our approach achieves a mean dice score of 80.41%, outperforming other well-known techniques such as UNet, Fully-Connected Network (FCN), Pyramid Scene Parsing Network (PSP Net), and the recent Convolutional vision Transformer (CvT) model, which achieved mean dice scores of 78.57%, 74.57%, 73.45%, and 62.97% respectively, under the same training conditions and using the same training dataset.

Image-based automatic multiple-defect detection of urban utility tunnel using UUTNet

In-service urban utility tunnels (UUT) suffer from cracks, corrosion, and leakage defects, which rises the chance of major accidents. However, prevailing detection methods for UUT remain reliant on manual inspection and subjective judgment, or traditional image processing technologies, such methods may not be able to obtain accurate defect information. This study proposes a novel and effective network called UUTNet based on the constructed UUT dataset for defects detection. Considering that the UUT defects has a certain distribution correlation, the attention module is introduced to the Pyramid Scene Parsing Network to capture the relation. By adding the hybrid dilated convolution after the feature extraction layer, the receptive field is expanded to further extract global and local features. The performance of UUTNet was evaluated based on the metrics MIoU, F1-score, Accuracy, and robustness. Comparative experiments were conducted, and the results showed the UUTNet achieved the best detection performance, achieving 0.7615 MIoU, 0.9806 Accuracy and 0.8012 F1-score. The MIoU was further improved to 0.7847 by utilizing the Bayesian optimization. Three extreme inspection scenes, including uneven illumination, high brightness, and obstacle interference, were applied to validate model robustness. The proposed method offers robust technical assistance for detecting defects in the UUT and precisely assessing the distribution and extent of these defects.