Pavement Crack Detection Research Articles

Automatic Pavement Crack Detection Fusing Attention Mechanism

Pavement cracks can result in the degradation of pavement performance. Due to the lack of timely inspection and reparation for the pavement cracks, with the development of cracks, the safety and service life of the pavement can be decreased. To curb the development of pavement cracks, detecting these cracks accurately plays an important role. In this paper, an automatic pavement crack detection method is proposed. For achieving real-time inspection, the YOLOV5 was selected as the base model. Due to the small size of the pavement cracks, the accuracy of most of the pavement crack deep learning-based methods cannot reach a high degree. To further improve the accuracy of those kind of methods, attention modules were employed. Based on the self-building datasets collected in Linyi city, the performance among various crack detection models was evaluated. The results showed that adding attention modules can effectively enhance the ability of crack detection. The precision of YOLOV5-CoordAtt reaches 95.27%. It was higher than other conventional and deep learning methods. According to the pictures of the results, the proposed methods can detect accurately under various situations.

Pavement Cracks Segmentation Algorithm Based on Conditional Generative Adversarial Network.

In long-term use, cracks will show up on the road, delivering monetary losses and security hazards. However, the road surface with a complex background has various disturbances, so it is challenging to segment the cracks accurately. Therefore, we propose a pavement cracks segmentation method based on a conditional generative adversarial network in this paper. U-net3+ with the attention module is used in the generator to generate segmented images for pavement cracks. The attention module highlights crack features and suppresses noise features from two dimensions of channel and space, then fuses the features generated by these two dimensions to obtain more complementary crack features. The original image is stitched with the manual annotation of cracks and the generated segmented image as the input of the discriminator. The PatchGAN method is used in the discriminator. Moreover, we propose a weighted hybrid loss function to improve the segmentation accuracy by exploiting the difference between the generated and annotated images. Through alternating gaming training of the generator and the discriminator, the segmentation image of cracks generated by the generator is very close to the actual segmentation image, thus achieving the effect of crack detection. Our experimental results using the Crack500 datasets show that the proposed method can eliminate various disturbances and achieve superior performance in pavement crack detection with complex backgrounds.

Data Augmentation by an Additional Self-Supervised CycleGAN-Based for Shadowed Pavement Detection

With the rapid development of deep learning, pavement crack detection has started to shift from traditional manual visual inspection to automated detection; however, automatic detection is still a challenge due to many complex interference conditions on pavements. To solve the problem of shadow interference in pavement crack detection, this paper proposes an improved shadow generation network, named Texture Self-Supervised CycleGAN (CycleGAN-TSS), which can improve the effect of generation and can be used to augment the band of shadowed images of pavement cracks. We selected various images from three public datasets, namely Crack500, cracktree200, and CFD, to create shadowed pavement-crack images and fed them into CycleGAN-TSS for training to inspect the generation effect of the network. To verify the effect of the proposed method on crack segmentation with shadow interference, the segmentation results of the augmented dataset were compared with those of the original dataset, using the U-Net. The results show that the segmentation network achieved a higher crack recognition accuracy after the augmented dataset was used for training. Our method, which involves generating shadowed images to augment the dataset and putting them into the training network, can effectively improve the anti-shadow interference ability of the crack segmentation network. The research in this paper also provides a feasible method for improving detection accuracy under other interference conditions in future pavement recognition work.

A Detection Method for Pavement Cracks Combining Object Detection and Attention Mechanism

Several deep learning techniques have been used to detect pavement cracks for the partial replacement of inefficient traditional inspections. However, the extensively varying real-world situations limit the detection accuracy. While many existing studies utilized the attention modules in pavement crack detection to improve model performance, few studies considered the impact of “how” and “where” to add attention modules on model performance, namely the module optimization. Combined with the attention mechanism, a new pavement crack detection method was proposed based on the You Only Look Once 5th version (YOLOv5) in this paper. Considering two adding ways and three adding positions, the spatial and channel squeeze and excitation (SCSE) module and convolutional block attention module (CBAM) were used to build a total of 12 different attention models for the cracking detection. Each model was trained on 3248 images, and the weight with the best performance on the validation set was saved for testing. The test results show that the mAP@0.5:0.95 of the best attention model is improved by nearly 6.7% compared to the original model without the attention mechanism. In addition, it can process images at 13.15ms/pic while maintaining 94.4% precision, fully meeting the needs of real-time detection. Compared with the existing pavement crack detection methods, the advantages of the proposed method include a great detection speed, high accuracy, and good robustness.

Automatic Detection and Counting System for Pavement Cracks Based on PCGAN and YOLO-MF

The regular detection of pavement cracks is critical for life and property security. However, existing deep learning-based methods of crack detection face difficulties in terms of data acquisition and defect counting. An automatic intelligent detection and tracking system for pavement cracks is proposed. Our system is formed of a pavement crack generative adversarial network (PCGAN) and a crack detection and tracking network called YOLO-MF. First, PCGAN is used to generate realistic crack images, to address the problem of the small number of available images. Next, YOLO-MF is developed based on an improved YOLO v3 modified by an acceleration algorithm and median flow (MF) algorithm to count the number of cracks. In a counting loop, our improved YOLO v3 detects cracks and the MF algorithm tracks the cracks detected in a video. This improved algorithm achieves the best accuracy of 98.47% and F1 score of 0.958 among other algorithms, and the precision-recall curve was close to the top right. A tiny model was developed and an acceleration algorithm was applied, which improved the detection speed by factors of five and six, respectively. In on-site measurement, three cracks were detected and tracked, and the total count was correct. Finally, the system was embedded in an intelligent device consisting of a calculating module, an automated unmanned aerial vehicle, and other components.

Asphalt Pavement Crack Detection Based on Convolutional Neural Network and Infrared Thermography

Two issues exist in the convolutional neural network (CNN) used for asphalt pavement crack detection: balance between accuracy and complexity, and indistinct edges of cracks and asphalt pavement surface. Infrared thermography (IRT) could use the temperature difference between cracks and pavement surface to better distinguish cracks. This work aims to propose a robust crack detection method based on CNN and IRT. An open benchmark dataset was built for crack detection based on three types of images, including visible images, infrared images, and the fusion of visible and infrared images (in short, fusion image). The dataset also considers different conditions and periods, including single, multi, thin, and thick cracks; clean, rough, light, and dark backgrounds, and three periods in a day. Seven CNN segmentation models are trained and evaluated on this dataset. To keep a balance between accuracy and complexity, evaluation metrics (accuracy, and computational and model complexity) are used to have an overall evaluation of models rather than only the accuracy. The results show that the accuracy and predictions of the visible image and fusion image are almost identical for all models, which are much better than that of the infrared image. When the background is rough or cracks are similar to the background, the fusion image is a better choice for crack detection. Compared with the visible image, all segmentation models have a more stable performance for the fusion image. Among segmentation models, Feature Pyramid Networks (FPN) could be the best model because of its high accuracy and low complexity.

Feature representation improved Faster R-CNN model for high-efficiency pavement crack detection

Two optimization methods are proposed to improve faster region-based convolutional neural network (Faster R-CNN), which are (1) restructuring Faster R-CNN's backbone network and the classification and regression (C&R) network using residual networks and (2) designing the feature ensemble structure for Faster R-CNN to combine the shallow with deep feature maps of the backbone. In addition, this paper proposed a method to evaluate the model's performance, which is pixel mean value ( Pmean) distribution of different channel feature maps, and quantitatively evaluate the feature representation capability of the model. Experimental results show that mean average precision (mAP) of the model optimized by the first method can reach 86.5%, which is 1.9% higher than that of baseline. However, mAP of the model optimized by the second method reaches 87.5%, which is 2.9% higher than the baseline model. The Pmean statistics of each channel feature map extracted by different backbones show that the model accuracy is higher when the Pmean of its channel feature maps is bigger, which can effectively improve the interpretability of the model accuracy.

Investigation of pavement crack detection based on deep learning method using weakly supervised instance segmentation framework

This paper presents efficient and cost-effective methods to identify pavement crack distress and thereby substantially increase pavement strength. Detecting the origin of this distress is the key to restoring pavement performance. To do that, a deep learning method is used to detect cracks based on the weakly supervised instance segmentation (WSIS) framework. A bounding box-level crack image data is preprocessed. Pseudo labels are generated by a region growing algorithm and a GrabCut algorithm. Another important contribution is a new dynamically balanced binary cross-entropy loss function. Results show that the WSIS framework reduces manual marking and has a high recognition accuracy of crack distress.

Applications of Artificial Intelligence Enhanced Drones in Distress Pavement, Pothole Detection, and Healthcare Monitoring with Service Delivery

Artificial Intelligence (AI) has fascinated the present study assigned to multiple areas such as distress detection on the pavement, pothole detection, and healthcare. The distress detection on pavement and roads and delivering healthcare and medical services need to be monitored through state-of-the-art technology, i.e., drone technology. Improvement in construction sites and healthcare delivery are of serious concern. Nowadays, computer vision techniques are commonly used in this area utilizing images and videos of construction sites. Due to confined data, researchers are using Unmanned Aerial Vehicles (UAVs) or Drone to get maximum information through 360° monitoring. This review article presents the useful monitoring techniques using AI-enabled drones for scholars around the world. In this comprehensive review, initially, the image acquisition equipment along with the perks and limitations has been presented. Second, the main constraints related to different computer vision techniques are highlighted for detecting distress in the pavement. Then, the possible research solution to some of the distress issues such that detection of pavement texture, cracks or potholes, joint faulting, temperature segregation, and rutting issues are predicted. Finally, the application of AI-enhanced drones in the healthcare field is elucidated which showed their significance. Moreover, in this research, the comparative image analysis of pavement and path hole detection was presented for the collection of detailed information and accurate detection. In the future, the work can also be enhanced to monitor the live pavement distress detection, especially for busy roads and highways. Moreover, an analysis to determine and reduce the costs in the healthcare sectors and organizations is required for future work.

Performance Analysis of the YOLOv4 Algorithm for Pavement Damage Image Detection with Different Embedding Positions of CBAM Modules

One of the most critical tasks for pavement maintenance and road safety is the rapid and correct identification and classification of asphalt pavement damages. Nowadays, deep learning networks have become the popular method for detecting pavement cracks, and there is always a need to further improve the accuracy and precision of pavement damage recognition. An improved YOLOv4-based pavement damage detection model was proposed in this study to address the above problems. The model improves the saliency of pavement damage by introducing the convolutional block attention module (CBAM) to suppress background noise and explores the influence of the embedding position of the CBAM module in the YOLOv4 model on the detection accuracy. The K-means++ algorithm was used to optimize the anchor box parameters to improve the target detection accuracy and form a high-performance pavement crack detection model called YOLOv4-3. The training and test sets were constructed using the same image data sources, and the results showed the mAP (mean average precision) of the improved YOLOv4-3 network was 2.96% higher than that before the improvement. The experiments indicate that embedding CBAM into the Neck module and the Head module can effectively improve the detection accuracy of the YOLOv4 model.

DMA-Net: DeepLab With Multi-Scale Attention for Pavement Crack Segmentation

Cracks are important indicators of pavement structural and operational conditions. Early pavement crack detection and treatments can help extend pavement service life, reduce fuel consumption, and improve safety and ride quality. Pavement distress surveys have traditionally been performed manually by visually inspecting the roads, which is labor-intensive and time-consuming. Therefore, computer-vision-based automated crack detection has great practical significance in pavement maintenance and traffic safety. Traditional image processing techniques are sensitive to noise in images and are thus likely to miss detecting some cracks due to the crack texture variety, complex lighting conditions, and various similar but irrelevant objects on the road. This paper adopts and enhances DeepLabv3+, a popular deep learning framework for semantic image segmentation, for road pavement crack detection. We propose a multi-scale attention module in the decoder of DeepLabv3+ to generate an attention mask and dynamically assign weights between high-level and low-level feature maps. Compared with fixed weights across different features, the dynamic weights strategy can assign more reasonable weights to different feature maps. Ablation experiments show that the attention mask can effectively help the model better combine multi-scale features and generate more accurate pavement crack segmentation results. The proposed method achieves state-of-the-art results on three benchmarks, including Crack500, DeepCrack, and FMA (Fitchburg Municipal Airport) datasets. We further test it on pavement crack images captured by smartphones, and the results show that it provides a viable approach to road pavement crack segmentation in practice with excellent performance.

A Method of Hierarchical Feature Fusion and Connected Attention Architecture for Pavement Crack Detection

Automatically detecting cracks with uneven strength from a complex background is a valuable and challenging issue. In light of the lost details and the incomplete extracted cracks in the process of crack extraction, we propose a network model with hierarchical feature fusion and connected attention architecture. Firstly, we build the backbone network on the improved DCA-SE-ResNet-50. Then, we propose a method for crack feature fusion, which combines depthwise separable convolution and dilated convolution to recover more crack details. Finally, we design the attention layer which integrates feature map2 with feature map4. The side network incorporates the feature maps of the low convolutional layer and the high convolutional layer at multiple levels to assist in obtaining the final prediction map. Sufficient experimental results demonstrate that our method achieved state-of-the-art performances, best F-score over 0.86, 12 FPS. Besides the effectiveness of our proposed method is verified on CFD, Crack500, and DCD datasets.

A Crack Detection Algorithm for Concrete Pavement Based on Attention Mechanism and Multi-Features Fusion

Crack detection for concrete pavement is an important and fundamental task to ensure road safety. However, automatic crack detection is a challenging topic due to the complicated concrete pavement background and the diversity of cracks. Inspired by the latest developments of deep learning in computer vision, we propose a novel crack detection algorithm of concrete pavement based on attention mechanism and multi-features fusion, and make it possible to deal with various cracks in different pavement backgrounds. The proposed network is constructed using the encoder-decoder structure. The architecture of the encoder part is consisted of Res2Net modules with attention mechanism to achieve fast focus of cracks. Cascade and parallel mode dilated convolutions are set as the center part to enlarge the receptive field of feature points without reducing the resolution of the feature maps. The decoder integrates multiple side output feature maps for pavement crack detection in the manner of feature pyramid. We use ODS, OIS and AP to evaluate the performance of our network. To demonstrate the validity and accuracy of the proposed method, we compare it with some existing methods. The experimental results in multiple crack datasets reveal that our method is superior to these methods.

GCN-Based Pavement Crack Detection Using Mobile LiDAR Point Clouds

Mobile Laser Scanning (MLS) system can provide high-density and accurate 3D point clouds that enable rapid pavement crack detection for road maintenance tasks. Supervised learning-based algorithms have been proved pretty effective for handling such a large amount of inhomogeneous and unstructured point clouds. However, these algorithms often rely on a lot of annotated data, which is labor-intensive and time-consuming. This paper presents a semi-supervised point-level approach to overcome this challenge. We propose a graph-widen module to construct a reasonable graph structure for point clouds, increasing the detection performance of graph convolutional networks (GCN). The constructed graph characterizes the local features from a small amount of annotated data, avoiding information loss and dramatically reduces the dependence on annotated data. The MLS point clouds acquired by a commercial RIEGL VMX-450 system are used in this study. The experimental results demonstrate that our method outperforms the state-of-the-art point-level methods in terms of recall, F1 score, and efficiency while achieving comparable accuracy.

MSK-UNET: A Modified U-Net Architecture Based on Selective Kernel with Multi-Scale Input for Pavement Crack Detection

Pavement crack condition is a vitally important indicator for road maintenance and driving safety. However, due to the interference of complex environment, such as illumination, shadow and noise, the automatic crack detection solution cannot meet the requirements of accuracy and efficiency. In this paper, we present an extended version of U-Net framework, named MSK-UNet, for pavement crack to solve these challenging problems. Specifically, first, the U-shaped network structure is chosen as the framework to extract more hierarchical representation. Second, we introduce selective kernel (SK) units to replace U-Net’s standard convolution blocks for obtaining the receptive fields with distinct scales. Third, multi-scale input layer establishes an image pyramid to retain more image context information at the encoder stage. Finally, a hybrid loss function including generalized Dice loss with Focal loss is employed. In addition, a regularization term is defined to reduce the impact of imbalance between positive and negative samples. To evaluate the performance of our algorithm, some tests were conducted on DeepCrack dataset, AsphaltCrack300 dataset and Crack500 dataset. Experimental results show that our approach can detect various crack types with diverse conditions, obtains a better performance in precision, recall and [Formula: see text]-score, with 97.43%, 96.95% and 97.01% precision values, 82.51%, 93.33% and 87.58% recall values and 95.33%, 99.24% and 98.55% [Formula: see text]-score values, respectively.

An Improved YOLOv5 Crack Detection Method Combined With Transformer

Efficient detection of pavement cracks can effectively prevent traffic accidents and reduce pavement maintenance costs. In order to overcome the complicated and uneconomical disadvantages of traditional crack detection methods, this paper introduces a pavement crack detection network based on deep learning, which can automatically detect pavement cracks and achieves excellent detection accuracy. And the network can easily use the sensors to collect data to facilitate industrial applications. In additional, considering that most cracks have slim feature, we apply the latest Transformer module in the network to improve the effect of cracks detection. Transformer has a strong ability to capture the long-range dependence of the cracks, which enables the network to learn the context information of the crack region. Furthermore, the network also utilizes some techniques to improve the ability of algorithm to detect various cracks. Our network is trained on pavement data sets containing India, the Czech Republic and Japan. It achieved F1 scores of 0.6739 and 0.6650 on two online test sets with fewer network parameters.

RIIAnet: A Real-Time Segmentation Network Integrated with Multi-Type Features of Different Depths for Pavement Cracks

Considerable road mileage puts tremendous pressure on pavement crack detection and maintenance. In practice, using a small parameter model for fast and accurate image-based crack segmentation is a challenge. However, current mainstream convolutional neural networks allocate computing resources to the same type of operators, which ignores the impact of different levels of feature extractors on the model performance. In this research, an end-to-end real-time pavement crack segmentation network (RIIAnet) is designed to improve performance by deploying different types of operators in separate layers of the network structure. Based on the extraction characteristics of cracks by convolution, involution, and asymmetric convolution, in the shallow layers the crack segmentation task is matched to extract rich low-level features by the designed asymmetric convolution enhancement module (ACE). Meanwhile, in the deep layers, the designed residual expanded involution module (REI) is used to enhance the high-level semantic features. Furthermore, the existing involution operator that fails to converge during training is improved. The ablation experiment demonstrates that the optimal ratio of the convolution and REI is 1/3 to obtain the optimal resource allocation and ACE improves the performance of the model. Especially compared with seven classical deep learning models of different structures, the results show that the proposed model reaches the highest MIOU, MPA, Recall, and F1 score of 0.7705, 0.9868, 0.8047, and 0.8485, respectively. More importantly, the parameter size of the proposed model is dramatically reduced, which is 0.04 times that of U-Net. In practice, the proposed model can be implemented in images with a high resolution of 2048 × 1024 in real time.

Evaluation of Small Uncrewed Aircraft Systems Data in Airfield Pavement Crack Detection and Rating

Current practice for airport Pavement Management Program (PMP) inspection relies on visual surveys and manual interpretation of reports and sketches prepared by inspectors in the field to quantify pavement conditions using the Pavement Condition Index method set forth in ASTM D5340. In recent years, several attempts have been made, both by the industry and by airport operators, to use small Uncrewed (Unpersonned/Unmanned) Aircraft Systems (sUAS), or “drones,” to conduct various types of imaging and inspection of airport pavements. As part of a comprehensive study on the use of such sUAS to evaluate airfield pavement conditions, the objectives of this study were to assess the performance of various sUAS platforms and sensors in detecting and rating a subset of crack-based pavement distresses and to evaluate the use of a combination of different sUAS datasets to complement current methods used to support airport PMP. Two airports in Michigan were selected for sUAS data collection, and five sUAS platforms equipped with eight different sensors were flown at these airports at different altitudes to collect red, green, and blue (RGB) optical and thermal data at different resolutions. RGB orthophotos, digital elevation models, and thermal images were visually analyzed to study their usefulness in detecting and rating longitudinal and transverse cracks in flexible/asphalt pavements and longitudinal, transverse, and diagonal cracks, corner breaks, and durability cracks in rigid/concrete pavements. This study demonstrated the capability of using sUAS data in detecting and rating multiple crack-related distresses in both flexible and rigid airfield pavement systems.

Automatic pavement crack detection using multimodal features fusion deep neural network

ABSTRACT The existing pavement crack detection algorithms are all dealing with detection and segmentation separately, ignoring the feature correlation between the bounding box coordinates and mask information, and there are still problems such as low crack detection accuracy, incomplete detection, and segmentation fracture in practical application. In view of the above situation, this paper took the detection bounding box coordinates and mask information as multimodal features of the same pavement crack area, and proposed a One-Stage MFFNet (Multimodal Feature Fusion Network), which improved the pavement crack detection accuracy and segmentation integrity obviously. And experimental results of different models were compared on self-collected datasets and two public datasets (CFD and CRACK500) respectively. Compared with Mask R-CNN, the average detection accuracy and average segmentation accuracy were improved by 2.6% and 4.7%, respectively. Compared with the optimised model of RDSNet, the detection accuracy and processing speed were improved by 1.8% and 2FPS, respectively. In addition, MFFNet had significantly improved the integrity of pavement crack segmentation results. The results showed that the proposed MFFNet model achieved the best detection and segmentation accuracy, and was an effective and high-precision pavement crack detection model.

The Improvement of Automated Crack Segmentation on Concrete Pavement with Graph Network

Crack is a common concrete pavement distress that will deteriorate into severe problems without timely repair, which means the automated detection of pavement crack is essential for pavement maintenance. However, automatic crack detection and segmentation remain challenging due to the complex pavement condition. Recent research on pavement crack detection based on deep learning has laid a good foundation for automated crack segmentation, but there can still be improvements. This paper proposes an automatic concrete pavement crack segmentation framework with enhanced graph network branch. First, the nodes of the graph and nodes’ attributions are generated based on the image dividing. The edges of the graph are determined based on Gaussian distribution. Then, the graph from the image is input into the graph branch. The graph feature map of the graph branch output is fused with the image feature map of the encoder and then enters the decoder to recover the image resolution to obtain the crack segmentation result. Finally, the method is tested on a self-built 3D concrete pavement crack dataset. The proposed method achieves the highest F1 and IoU (Intersection over Union) in the comparison experiments. And the graph branch addition improves 0.08 on F1 and 0.06 on IoU compared with U-Net.