Automated Pavement Crack Detection Research Articles

Pavement Crack Detection and Identification Based on Improved YOLOv8

Aiming at the problem of poor real-time performance and low precision of traditional pavement crack detection, an improved YOLOv8 algorithm is proposed to realize the automatic pavement crack detection and identification. Firstly, the crack image is manually labeled by LabelImg labeling software, and then the model parameters are obtained by improving YOLOv8 network. The improved method includes introducing a convolution block attention module (CBAM) to enhance the feature extraction ability, and optimizing the weighted intersection-to-union ratio (Wiou) loss function. Through comprehensive evaluation indexes (F1-measure and mAP@50-95%), the performance of the original model and the improved model in pavement crack detection and recognition is compared. The results show that the improved model has improved the detection precision, F1 value and mAP@50-95% value, which are 2.91%, 3.29% and 1.7% respectively. These results show that the improved model has significant practical significance in realizing the automatic identification of pavement cracks.

A generative adversarial network approach for removing motion blur in the automatic detection of pavement cracks

AbstractAdvancements in infrastructure management have significantly benefited from automatic pavement crack detection systems, relying on image processing enhanced by high‐resolution imaging and machine learning. However, image and motion blur substantially challenge the accuracy of crack detection and analysis. Nevertheless, research on mitigating motion blur remains sparse. This study introduces an effective image processing system adept at deblurring and segmentation, employing a generative adversarial network (GAN) with UNet as the generator and Wasserstein GAN with Gradient Penalty (WGAN‐gp) as the loss function. This approach performs exceptionally in deblurring pavement crack images and improves segmentation accuracy. Models were trained with sharp and artificially blurred images, with WGAN‐gp surpassing other loss functions in effectiveness. This research innovatively suggests assessing deblurring quality through segmentation accuracy in addition to peak signal‐to‐noise ratio (PSNR) and structural similarity (SSIM), revealing that PSNR and SSIM may not fully capture deblurring effectiveness for pavement crack images. An extensive evaluation of various generators, including UNet, lightweight UNet, TransUNet, DeblurGAN, DeblurGAN‐v2, and MIMO‐UNet, identifies the superior performance of UNet on simulated motion blur. Validation with actual motion‐blurred images confirms the effectiveness of the proposed model. These findings demonstrate that GAN‐based models have great potential in overcoming motion blur challenges in pavement crack detection systems, marking a notable advancement in the field.

A two-stage framework for pixel-level pavement surface crack detection

Surface crack is one of the most common distresses of pavement structure, impacting its serviceability and sustainability. Over the past decade, many efforts have been devoted to developing computer vision-based models (e.g., image processing- or deep learning-based) for the automatic detection of pavement surface crack. However, there is a great gap between the public image data taken by the phone or other portable devices and the real-world image data acquired by the linear array charge-coupled device (CCD) camera, which usually is high resolution and contains limited crack pixels per image. To improve the pavement surface crack detection efficiency and accuracy in engineering practice, we propose a novel two-stage framework for automatic pavement surface detection at the pixel level. In stage I, the images concluding pixel cracks are selected using a convolutional neural network (CNN)-based classification network. In stage II, the selected images are processed with our proposed separation-combination strategy and the second version of crack transformer (CTv2) for pavement surface crack detection at the pixel level. Comprehensive experimental investigation and comparison have been conducted on training performance and visualization results, validating the superiority of the developed framework. It paves the way for the large-scale application of automatic pavement crack detection in an efficient manner.

Automatic Pavement Crack Detection in Multisource Fusion Images Using Similarity and Difference Features

In urban pavement crack detection, the single detection platform with visible light sensors is mostly used for detection. Visible light sensors are highly susceptible to lighting conditions. In low-light environments or areas with shadows over the cracks, this may result in false positives or missed detections of pavement cracks. The paper utilizes an image acquisition system that integrates multiple detection platforms, including visible light and infrared sensors, to enhance both the efficiency and accuracy of pavement crack detection. We propose an automatic pavement crack detection approach that uses the similarities and differences between visible light images and infrared image features (MSFSD) to achieve all-weather, real-time detection of pavement cracks. MSFSD mainly includes multi-source image fusion and pixel-level crack detection. In multi-source image fusion, a generative adversarial network (C-GAN) with skip connections and dilated convolutional blocks (CRes2Net blocks) is constructed. The structural similarity index (SSIM) function and the sum of the correlations of differences (SCD) function are used as the loss function of the generator, allowing the fusion image to retain more details and edge information in the source image. In pixel-level crack detection, a crack detection model is constructed, and CRes2Net is used as the core network to obtain pixel-level labels and perform semantic segmentation. It is verified by experiments that MSFSD achieves 85.1% precision and 88.4% recall in different types of crack detection. The MIoU and detection efficiency of MSFSD are 9% and 48% higher than other methods, respectively.

An attention-based progressive fusion network for pixelwise pavement crack detection

Detecting pavement cracks is a complex task in pavement maintenance. Convolutional neural network (CNN)-based approaches have been increasingly used for this purpose in recent years, but these approaches still have limitations, such as insufficient processing of local features, poor boundary detection ability and information loss issue. In this paper, a novel CNN-based model, named APF-Net, is presented. Specifically, to extract richer and more feature information, proposed model utilizes a progressive fusion module (PF) that enables effective feature enhancement by feature learning from cross-layer features, which operates progressively across adjacent network layers, enabling the crack segmentation model to extract more global and detail features from the crack images. Additionally, a hybrid multiple attention (HMA) module that incorporates both a spatial attention mechanism from two directions and a channel attention mechanism, is proposed, which could effectively capture the long-term dependence of crack features and enhance crack boundary detection. Moreover, proposed model employs a lightweight feature extraction block, named mobile inverted bottleneck convolution (MBC) block, which significantly reduces the model parameters and enables more efficient processing of the crack images. This approach results in a reduction of the computational load and faster detection of pavement cracks. The effectiveness and generalization of the proposed APF-Net have been evaluated through experiments on three publicly available crack datasets. The results demonstrate that the proposed model outperforms other state-of-the-art segmentation models in detecting pavement cracks while with 7.6M parameters and 5.0G FLOPs. Our proposed model can contribute to the development of automatic pavement crack detection systems, which can aid in the maintenance and preservation of roads and highways. The source code for proposed APF-Net is available at https://github.com/MMYZZU/APFNet.git.

Encoder–decoder with pyramid region attention for pixel‐level pavement crack recognition

AbstractTimely and accurate extraction of pavement crack information is crucial to maintain service conditions and structural safety for infrastructures and reduce further road maintenance costs. Currently, deep learning techniques for automated pavement crack detection are far superior to traditional manual approaches in both speed and accuracy. However, existing deep learning models may easily lose crack details when processing images containing complex background textures or other noises. Although many studies have alleviated this challenge by introducing attention mechanisms, especially the non‐local (NL) block, which has the ability to efficiently capture long‐range dependencies to facilitate crack pixel capture, the huge computational cost of NL makes the inference time of the model too long, which is not conducive to practical implementation. In this study, a new module, namely, the pyramid region attention module (PRAM), was developed by combining the pyramid pooling module in the pyramid scene parsing network and optimized NL, which can achieve global multi‐scale context integration and long‐range dependencies capture at a relatively lower computational cost. By applying PRAM to deep skip connections in the modified U‐Net, an effective crack segmentation model called CrackResU‐Net was developed. The test results on the existing CrackForest dataset showed that CrackResU‐Net not only achieved an F1 score of 0.9580 but also took only 25.89 ms to process an image with a resolution of 480 × 320, which had advantages in accuracy and speed, compared with several other state‐of‐the‐art crack segmentation approaches. It was fully demonstrated that this approach could realize automatic fast and high‐precision recognition of pavement cracks for engineering purposes.

Crack Segmentation Extraction and Parameter Calculation of Asphalt Pavement Based on Image Processing.

Crack disease is one of the most serious and common diseases in road detection. Traditional manual methods for measuring crack detection can no longer meet the needs of road crack detection. In previous work, the authors proposed a crack detection method for asphalt pavements based on an improved YOLOv5s model, which is a better model for detecting various types of cracks in asphalt pavements. However, most of the current research on automatic pavement crack detection is still focused on crack identification and location stages, which contributes little to practical engineering applications. Based on the shortcomings of the above work, and in order to improve its contribution to practical engineering applications, this paper proposes a method for segmenting and analyzing asphalt pavement cracks and identifying parameters based on image processing. The first step is to extract the crack profile through image grayscale, histogram equalization, segmented linear transformation, median filtering, Sauvola binarization, and the connected domain threshold method. Then, the magnification between the pixel area and the actual area of the calibration object is calculated. The second step is to extract the skeleton from the crack profile images of asphalt pavement using the Zhang-Suen thinning algorithm, followed by removing the burrs of the crack skeleton image using the connected domain threshold method. The final step is to calculate physical parameters, such as the actual area, width, segments, and length of the crack with images obtained from the crack profile and skeleton. The results show that (1) the method of local thresholding and connected domain thresholding can completely filter noise regions under the premise of retaining detailed crack region information. (2) The Zhang-Suen iterative refinement algorithm is faster in extracting the crack skeleton of asphalt pavement, retaining the foreground features of the image better, while the connected-domain thresholding method is able to eliminate the missed isolated noise. (3) In comparison to the manual calibration method, the crack parameter calculation method proposed in this paper can better complete the calculation of crack length, width, and area within an allowable margin of error. On the basis of this research, a windowing system for asphalt pavement crack detection, WSPCD1.0, was developed. It integrates the research results from this paper, facilitating automated detection and parameter output for asphalt pavement cracks.

A lightweight encoder–decoder network for automatic pavement crack detection

AbstractCracks are the most common damage type on the pavement surface. Usually, pavement cracks, especially small cracks, are difficult to be accurately identified due to background interference. Accurate and fast automatic road crack detection play a vital role in assessing pavement conditions. Thus, this paper proposes an efficient lightweight encoder–decoder network for automatically detecting pavement cracks at the pixel level. Taking advantage of a novel encoder–decoder architecture integrating a new type of hybrid attention blocks and residual blocks (RBs), the proposed network can achieve an extremely lightweight model with more accurate detection of pavement crack pixels. An image dataset consisting of 789 images of pavement cracks acquired by a self‐designed mobile robot is built and utilized to train and evaluate the proposed network. Comprehensive experiments demonstrate that the proposed network performs better than the state‐of‐the‐art methods on the self‐built dataset as well as three other public datasets (CamCrack789, Crack500, CFD, and DeepCrack237), achieving F1 scores of 94.94%, 82.95%, 95.74%, and 92.51%, respectively. Additionally, ablation studies validate the effectiveness of integrating the RBs and the proposed hybrid attention mechanisms. By introducing depth‐wise separable convolutions, an even more lightweight version of the proposed network is created, which has a comparable performance and achieves the fastest inference speed with a model parameter size of only 0.57 M. The developed mobile robot system can effectively detect pavement cracks in real scenarios at a speed of 25 frames per second.

Synergizing Low Rank Representation and Deep Learning for Automatic Pavement Crack Detection

Due to the critical role of pavement crack detection for road maintenance and eventually ensuring safety, remarkable efforts have been devoted to this research area, and such a trend is further intensified for the coming unmanned vehicle era. However, such crack detection task still remains unexpectedly challenging in practice since the appearance of both cracks and the background are diverse and complex in real scenarios. In this work, we propose an automatic pavement crack detection method via synergizing low rank representation (LRR) and deep learning techniques. First, leveraging LRR which facilitates anomaly detection without making any specific assumption, we can easily discriminate most of the frames with cracks from the long sequence with a consistent pavement base, followed by a straightforward algorithm to localize the cracks. In order to achieve the intelligence of detecting cracks with different pavement basis under unconstrained imaging conditions, we resort to deep learning techniques and propose a deep convolutional neural network for crack detection leveraging on multi-level features and atrous spatial pyramid pooling (ASPP). We train this network based on the training data obtained in the previous stage in an end-to-end manner. Extensive experiments on a wide range of pavements demonstrate the high performance in terms of both accuracy and automaticity. Moreover, the dataset generated by us is much more extensive and challenging than public ones. We put it online at https://gaozhinuswhu.com to benefit the community.

Automated pavement crack detection based on multiscale fully convolutional network

AbstractAutomatic pavement crack detection is essential for fast and efficient pavement maintenance and health measurement. And crack image data is the basis of crack detection. The existing data collection methods have disadvantages such as high cost, easy loss of frames, blurring, and loss of crack information. Therefore, a new method of data collection using target detection and perspective transformation is introduced. The CRACK2000 dataset with multiple complex backgrounds is constructed by this method. Also, a multiscale fully convolutional network by improving U‐Net, named U‐multiscale dilated network (U‐MDN), is proposed. The network uses U‐Net as the base network and introduces U‐multiscale dilated convolutional module (U‐MDM) after U‐Net downsampling. In addition, the U‐MDM is compared with U‐MCM and MDM, and the result shows that U‐MDM has a better effect. Finally, U‐MDN is compared with U‐Net, CrackSeg, DeeplabV3+, Basnet, and PDDF‐Net on CRACK2000 and other data sets, respectively. The experimental results demonstrate that the U‐MDN is better than other algorithms in terms of precision, recall, F1‐score, and AUC.

Automated crack segmentation on 3D asphalt surfaces with richer attention and hybrid pyramid structures

ABSTRACT Automated pavement crack detection is crucial to supporting fine pavement maintenance and ensuring safety for road facilities. Due to the complex pavement condition and crack features, it is still a critical challenge in intelligent pavement surveys. This paper proposed a novel pixel-level pavement crack segmentation network, PCSNet, to provide a solution to this challenge. The network has richer attention and hybrid pyramid structures, which implement full-process crack feature fusion and enhancement. The richer attention module consists of cascaded self-attention and attention gate modules. It captures the crack spatial dependence information and prunes the feature response. The hybrid pyramid structures consist of a multistage convolutional pyramid module and a pyramid pooling module. It integrates contextual information at multiple receptive field scales to enhance the potential crack feature representation. The proposed structure enriches the crack details and optimises the scene parsing on the global geometry of the cracks. A sizeable 3D pavement crack dataset is built for training and testing. The proposed network exhibited the best performance, achieving F1-score, mean intersection of union, and mean pixel accuracy of 81.21%, 77.13%, and 87.17%, respectively. The network can reconstruct the complete crack geometry, preserve the crack edges well, and optimises the detection of shallow and complex cracks. The method exhibits superior and robust performance, facilitating accurate pavement technical condition assessment and maintenance decisions.

Deep Learning based Pavement Crack Detection System

The pavement crack causes the highway service life to shorten, the safety hidden danger to increase. The low efficiency and high cost of manual inspection makes it difficult to detect pavement cracks. This paper proposes a fast and efficient deep learning pavement crack detection system. CRACK2000, an image segmentation dataset with complex interference background and multiple crack types, is constructed based on perspective transformation and image cropping. The scheme corrects the pavement crack images by perspective transformation. The extraction of pavement crack depth features is completed by applying the U-Net network. Finally, the pavement condition index PCI (pavement condition index) is calculated by quantifying the different types of crack information based on the segmentation results. The experimental results show that the Precision, Recall, F1-score and AUC of the U-Net network are 76.67%, 72.32%, 74.43% and 99.46% respectively. The AUC values reflect that the method is more capable of filtering out complex background interference from cracked images. The automatic pavement crack detection system designed in this paper can accurately locate and classify the location and category of pavement cracks, and perform quantitative pavement evaluation to obtain the pavement deterioration of the road section and the corresponding repair recommendations, enhancing the practicality of pavement crack detection.

Pavement crack detection based on point cloud data and data fusion.

The three-dimensional detection in point cloud data for pavement cracks has drawn the attention of many researchers recently. In the field of pavement surface point cloud detection, the key tasks include the identification of pavement cracks and the extraction of the location and size information of pavement cracks. Based on the point cloud data of pavement surface, we developed two methods to directly extract and detect cracks, respectively. The first method is based on the improved sliding window algorithm by combining the random sample consensus (RANSAC) technique to directly extract the crack information from point clouds. The second method is developed based on YOLOv5 to process the two-dimensional images transformed from point cloud data for automatic pavement crack detection. We also attempted to fuse the point cloud images with greyscale images as input for the YOLOv5. Analysis results show that the improved sliding window algorithm efficiently extracts pavement cracks with less noise, and the YOLOv5-based method obtains a good detection of pavement cracks. This article is part of the theme issue 'Artificial intelligence in failure analysis of transportation infrastructure and materials'.

Pavement Cracks Coupled With Shadows: A New Shadow-Crack Dataset and A Shadow-Removal-Oriented Crack Detection Approach

Automatic pavement crack detection is a critical task for maintaining the pavement stability and driving safety. The task is challenging because the shadows on the pavement may have similar intensity with the crack, which interfere with the crack detection performance. Till to the present, there still lacks efficient algorithm models and training datasets to deal with the interference brought by the shadows. To fill in the gap, we made several contributions as follows. First, we proposed a new pavement shadow and crack dataset, which contains a variety of shadow and pavement pixel size combinations. It also covers all common cracks (linear cracks and network cracks), placing higher demands on crack detection methods. Second, we designed a two-step shadow-removal-oriented crack detection approach: SROCD, which improves the performance of the algorithm by first removing the shadow and then detecting it. In addition to shadows, the method can cope with other noise disturbances. Third, we explored the mechanism of how shadows affect crack detection. Based on this mechanism, we propose a data augmentation method based on the difference in brightness values, which can adapt to brightness changes caused by seasonal and weather changes. Finally, we introduced a residual feature augmentation algorithm to detect small cracks that can predict sudden disasters, and the algorithm improves the performance of the model overall. We compare our method with the state-of-the-art methods on existing pavement crack datasets and the shadow-crack dataset, and the experimental results demonstrate the superiority of our method.

A novel transformer-based network with attention mechanism for automatic pavement crack detection

Currently, there is an urgent need to utilize automatic approaches to detecting pavement cracks for roadway maintenance. Taking advantage of the development of convolutional neural networks (CNNs), previous studies put more effort into the detection of the pavement crack with local feature extraction using consecutive convolutional operations. However, it results in the loss of detailed information, making CNNs fail to accurately inspect the long and complicated cracks under noisy conditions, which are common on the pavement surface, negatively impacting detection accuracy. In order to cope with this issue, this study proposes a Transformer-based semantic segmentation network that unifies the Swin Transformer as the Encoder and the UperNet with the attention module as the Decoder for robust and accurate pixel-level pavement crack detection. Leveraging the hierarchical architecture of Swin Transformer, the global and long-range semantic features of the pavement crack are learned for improved segmentation accuracy. With the assistance of the attention module, the Decoder can retrieve more details of the crack information, presenting accurate detection results on the fine and tiny pavement cracks. To validate the superiority of the proposed network, we have trained and tested six semantic segmentation models on three public pavement crack datasets. Compared to other models, the proposed model achieves the best performance on visualization and evaluation metrics of mean F1(mF1) and mean Recall (mRecall) with 0-pixel tolerance. It paves the way for future applications of automatic pavement crack detection using Transformed-based networks.

Automated Pavement Crack Detection with Deep Learning Methods: What Are the Main Factors and How to Improve the Performance?

Recently, deep learning (DL) methods have been intensively studied in pavement distress detection research. To build an effective yet robust DL-based distress detection model, both the training data and DL model structure need to be carefully prepared. This paper discusses the main influencing factors that affect the performance of DL models in pavement distress detection and aims to explore possible ways to improve the model performance. An open-source pavement image dataset and three promising object detection methods were selected for model development and evaluation. A series of experiments were conducted with different settings of model structures and data manipulation. According to the experiment results, selection of detection methods, data size, and annotation consistency have significant impact on detection performance. DL models YOLO and Faster R-CNN were found to yield the same level of performance, while CenterNet notably underperformed on the test dataset. Distress classes with a larger number of instances were found to be able to yield better performances. With the new annotation data carefully marked up and verified by experienced pavement raters, the accuracy score using the F1 index was found to be 0.84, which was a very sound improvement compared with the 0.7 of the original annotation. Data augmentation methods were found to be able to significantly improve the detection accuracy. Based on the experiment results, we believe the focus should be placed on the development of high-quality training datasets and thin object detection methods in the future.

Automatic Pavement Crack Detection Transformer Based on Convolutional and Sequential Feature Fusion

To solve the problem of low accuracy of pavement crack detection caused by natural environment interference, this paper designed a lightweight detection framework named PCDETR (Pavement Crack DEtection TRansformer) network, based on the fusion of the convolution features with the sequence features and proposed an efficient pavement crack detection method. Firstly, the scalable Swin-Transformer network and the residual network are used as two parallel channels of the backbone network to extract the long-sequence global features and the underlying visual local features of the pavement cracks, respectively, which are concatenated and fused to enrich the extracted feature information. Then, the encoder and decoder of the transformer detection framework are optimized; the location and category information of the pavement cracks can be obtained directly using the set prediction, which provided a low-code method to reduce the implementation complexity. The research result shows that the highest AP (Average Precision) of this method reaches 45.8% on the COCO dataset, which is significantly higher than that of DETR and its variants model Conditional DETR where the AP values are 36.9% and 42.8%, respectively. On the self-collected pavement crack dataset, the AP of the proposed method reaches 45.6%, which is 3.8% higher than that of Mask R-CNN (Region-based Convolution Neural Network) and 8.8% higher than that of Faster R-CNN. Therefore, this method is an efficient pavement crack detection algorithm.

An automatic image processing based on Hough transform algorithm for pavement crack detection and classification

PurposeIncipient detection of pavement deterioration (such as crack identification) is critical to optimizing road maintenance because it enables preventative steps to be implemented to mitigate damage and possible failure. Traditional visual inspection has been largely superseded by semi-automatic/automatic procedures given significant advancements in image processing. Therefore, there is a need to develop automated tools to detect and classify cracks.Design/methodology/approachThe literature review is employed to evaluate existing attempts to use Hough transform algorithm and highlight issues that should be improved. Then, developing a simple low-cost crack detection method based on the Hough transform algorithm for pavement crack detection and classification.FindingsAnalysis results reveal that model accuracy reaches 92.14% for vertical cracks, 93.03% for diagonal cracks and 95.61% for horizontal cracks. The time lapse for detecting the crack type for one image is circa 0.98 s for vertical cracks, 0.79 s for horizontal cracks and 0.83 s for diagonal cracks. Ensuing discourse serves to illustrate the inherent potential of a simple low-cost image processing method in automated pavement crack detection. Moreover, this method provides direct guidance for long-term pavement optimal maintenance decisions.Research limitations/implicationsThe outcome of this research can help highway agencies to detect and classify cracks accurately for a very long highway without a need for manual inspection, which can significantly minimize cost.Originality/valueHough transform algorithm was tested in terms of detect and classify a large dataset of highway images, and the accuracy reaches 92.14%, which can be considered as a very accurate percentage regarding automated cracks and distresses classification.

STrans-YOLOX: Fusing Swin Transformer and YOLOX for Automatic Pavement Crack Detection

Automatic pavement crack detection is crucial for reducing road maintenance costs and ensuring transportation safety. Although convolutional neural networks (CNNs) have been widely used in automatic pavement crack detection, they cannot adequately model the long-range dependencies between pixels and easily lose edge detail information in complex scenes. Moreover, irregular crack shapes also make the detection task challenging. To address these issues, an automatic pavement crack detection architecture named STrans-YOLOX is proposed. Specifically, the architecture first exploits the CNN backbone to extract feature information, preserving the local modeling ability of the CNN. Then, Swin Transformer is introduced to enhance the long-range dependencies through a self-attention mechanism by supplying each pixel with global features. A new global attention guidance module (GAGM) is used to ensure effective information propagation in the feature pyramid network (FPN) by using high-level semantic information to guide the low-level spatial information, thereby enhancing the multi-class and multi-scale features of cracks. During the post-processing stage, we utilize α-IoU-NMS to achieve the accurate suppression of the detection boxes in the case of occlusion and overlapping objects by introducing an adjustable power parameter. The experiments demonstrate that the proposed STrans-YOLOX achieves 63.37% mAP and surpasses the state-of-the-art models on the challenging pavement crack dataset.

Automated Pavement Crack Detection Using Deep Feature Selection and Whale Optimization Algorithm

Automated Pavement Crack Detection Using Deep Feature Selection and Whale Optimization Algorithm