Pavement Images Research Articles

ARDs-YOLO: Intelligent detection of asphalt road damages and evaluation of pavement condition in complex scenarios

Pavement distress detection is the key to the development of road maintenance programs. However, traditional methods with long detection period and strong subjectivity are difficult to meet the rapid and large-scale automatic detection and evaluation of pavement technical condition. In this paper, we propose a method based on YOLOv5 combined with the initialized anchor frame selection strategy of intersection and merger ratio metrics, the multidimensional information interaction attention mechanism module ARDs, and the lightweight up-sampling operator CARAFE. An asphalt pavement distress dataset ARDs-5 containing 21,797 pavement images is constructed using a vehicle-mounted CCD camera. The proposed method outperforms the comparison frontier baseline on the self-constructed dataset with a mAP of 75.8%. Finally, ARD-YOLO was applied to evaluate the pavement condition of asphalt highways in Liaoning Province, China, and achieved excellent performance. The experiments show that ARD-YOLO has high practical value for pavement distress detection and evaluation in complex environments.

Application of the Semi-Supervised Learning Approach for Pavement Defect Detection.

Road surface quality is essential for driver comfort and safety, making it crucial to monitor pavement conditions and detect defects in real time. However, the diversity of defects and the complexity of ambient conditions make it challenging to develop an effective and robust classification and detection algorithm. In this study, we adopted a semi-supervised learning approach to train ResNet-18 for image feature retrieval and then classification and detection of pavement defects. The resulting feature embedding vectors from image patches were retrieved, concatenated, and randomly sampled to model a multivariate normal distribution based on the only one-class training pavement image dataset. The calibration pavement image dataset was used to determine the defect score threshold based on the receiver operating characteristic curve, with the Mahalanobis distance employed as a metric to evaluate differences between normal and defect pavement images. Finally, a heatmap derived from the defect score map for the testing dataset was overlaid on the original pavement images to provide insight into the network's decisions and guide measures to improve its performance. The results demonstrate that the model's classification accuracy improved from 0.868 to 0.887 using the expanded and augmented pavement image data based on the analysis of heatmaps.

Classification of different size of potholes based on surface area using convolutional neural network

Potholes are a significant hazard, causing severe damage to vehicles and potentially leading to fatal accidents. The automatic detection of potholes is crucial for timely maintenance and reducing these risks. This research evaluates the performance of three pre-trained Convolutional Neural Network models—ResNet 50, ResNet 18, and MobileNet—in classifying pavement images. Initially, the models distinguish between images containing potholes and those without (Potholes vs. Normal). Subsequently, they categorize pavement images into three classes: Small Pothole, Large Pothole, and Normal. Pavement images were captured from two different heights: 3.5 feet (waist height) and 2 feet. Among the models, MobileNet v2 demonstrated the highest accuracy of 98% for detecting potholes. For images taken at 2 feet, the classification accuracies for large potholes, small potholes, and normal pavement were 87.33%, 88.67%, and 92%, respectively. For images taken from waist height, the accuracies increased to 98.67%, 98.67%, and 100%, respectively. The study highlights the potential of these models in enhancing road safety through reliable and automated pothole detection, providing valuable insights for infrastructure maintenance.Article HighlightsPotholes are dangerous and can cause serious vehicle damage.Train CNN models for the classification of potholesThese CNNs, for different sizes of potholes can be used for the rehabilitation of potholes.Among all CNN modes, MobileNet v2 achieves 98% accuracy in detecting the presence of a pothole.

Detection of Road Risk Sources Based on Multi-Scale Lightweight Networks.

Timely discovery and disposal of road risk sources constitute the cornerstone of road operation safety. Presently, the detection of road risk sources frequently relies on manual inspections via inspection vehicles, a process that is both inefficient and time-consuming. To tackle this challenge, this paper introduces a novel automated approach for detecting road risk sources, termed the multi-scale lightweight network (MSLN). This method primarily focuses on identifying road surfaces, potholes, and scattered objects. To mitigate the influence of real-world factors such as noise and uneven brightness on test results, pavement images were carefully collected. Initially, the collected images underwent grayscale processing. Subsequently, the median filtering algorithm was employed to filter out noise interference. Furthermore, adaptive histogram equalization techniques were utilized to enhance the visibility of cracks and the road background. Following these preprocessing steps, the MSLN model was deployed for the detection of road risk sources. Addressing the challenges associated with two-stage network models, such as prolonged training and testing times, as well as deployment difficulties, this study adopted the lightweight feature extraction network MobileNetV2. Additionally, transfer learning was incorporated to elevate the model's training efficiency. Moreover, this paper established a mapping relationship model that transitions from the world coordinate system to the pixel coordinate system. This model enables the calculation of risk source dimensions based on detection outcomes. Experimental results reveal that the MSLN model exhibits a notably faster convergence rate. This enhanced convergence not only boosts training speed but also elevates the precision of risk source detection. Furthermore, the proposed mapping relationship coordinate transformation model proves highly effective in determining the scale of risk sources.

Automated pixel-level pavement marking detection based on a convolutional transformer

Accurate detection of pavement markings at the pixel level is crucial for enhancing traffic safety. The majority of current advanced deep-learning networks predominantly focus on localized features, neglecting the global context of pavement image. Such networks often result in discontinuous segmentation outcomes and suboptimal recovery of local details. In this paper, a robust model named C-Transformer is proposed to provide an effective solution to this challenge. The contributions of this paper primarily involve two aspects. Firstly, the proposed C-Transformer is designed to succinctly integrate convolution operations and self-attention, facilitating a comprehensive understanding of essential features. Secondly, an efficient Feed-Forward Network called Inverse Residual Feed-Forward Network is also proposed in this paper and deployed in C-Transformer to improve latent representations. Experimental results demonstrate that, compared to other state-of-the-art networks, the proposed C-Transformer achieves a performance enhancement of 0.93% in F-measure and a 1.64% improvement in Intersection-Over-Union. In particular, the robustness and effectiveness of the C-Transformer in accurate pavement marking detection are proved through field test results. This paper illustrates the feasibility of employing a hybrid Convolutional neural network-Transformer-based network for automatic robust pavement marking detection under noisy conditions.

Foreign Object Debris Detection for Airport Pavement Images Based on Image Random Masking and Semantic Feature Matching

Foreign Object Debris Detection for Airport Pavement Images Based on Image Random Masking and Semantic Feature Matching

Research on road surface crack detection based on SegNet network

To enhance the precision and reliability of road crack detection, this study introduces an innovative neural network architecture. Strategies were implemented to effectively address the issue of overfitting resulting from the intricacy of the proposed SegCrackNet. Dropout layers, multi-level output fusion, and T-bridge block structures are employed in the network. This optimization allows for a more comprehensive exploitation of contextual information, demonstrating its instrumental role in the efficient detection of subtle variations. Experimental findings clearly demonstrate substantial improvements when compared to other network models. On the Crack500, Crack200, and pavement images datasets, remarkable enhancements in the average Intersection over Union (IoU) scores were observed, with increases of 4.3%, 9.4%, and 3.7%, respectively.

Enhancing sustainability for pavement maintenance decision-making through image processing-based distress detection

Road maintenance sustainability entails implementing practices that reduce the impact of road accident. while simultaneously ensuring the durability and functionality of the road infrastructure. Pavement distress is a major concern for transportation agencies as it affects the safety and comfort of road users. This paper presents a novel approach for prioritizing pavement distress through the application of image processing techniques and the Analytic Hierarchy Process (AHP). The proposed method involves capturing images of the pavement surface using a high-resolution camera and analyzing them using image processing algorithms. The images are processed to identify different types of pavement distress such as cracks, potholes, and rutting. The severity of each type of distress is then quantified. AHP is utilized to prioritize the identified distress based on its severity and impact on road users. The proposed approach has been tested on real-world pavement images, and the results demonstrate its effectiveness in accurately identifying and prioritizing pavement distress. This method can assist transportation agencies in making informed decisions about maintenance and repair activities, leading to improved road safety and reduced maintenance costs. This automated method achieved an accuracy percentage of about 95%.

Multi-Objective Optimization for Sustainable Pavement Maintenance Decision Making by Integrating Pavement Image Segmentation and TOPSIS Methods

To provide a low-carbon economy maintenance strategy is the most challenging problem faced by pavement management authorities under the restricted budget and significant environmental repercussions. The development of a multi-objective optimization model for pavement maintenance decision making is essential to formulate pavements. Nevertheless, the existing automatic detection can only recognize and classify pavement distress. However, few studies are able to accurately determine the precise dimensions of specific distresses such as cracks and potholes, especially combined with the actual size of the image. This limitation hinders the ability to provide specific maintenance recommendations and make optimal maintenance decisions. Therefore, this paper develops a comprehensive and effective multi-objective decision-making framework for pavement maintenance. This framework consists of four distinct components: (1) recognizing the dimensions of pavement distresses based on the pavement image segmentation technique; (2) compiling a list of viable pavement maintenance strategies; (3) assessing the costs and carbon emissions of these strategies; and (4) optimizing decisions on pavement maintenance. We used the U-Net algorithm to accurately recognize the dimensions of pavement distresses, while an improved entropy-weighted TOPSIS model was proposed to determine the optimal pavement maintenance strategy with the lowest cost and carbon emissions. The results indicated that the pavement distress dimension recognition model achieved a high accuracy of 96.88%, and the TOPSIS model identified the optimal maintenance strategy with a score of 99.16. This maintenance strategy achieved a substantial reduction of 30.80% in carbon emissions and a cost reduction of 20.81% compared to the highest values among all maintenance strategies. This study not only provides a scientifically objective method for making pavement maintenance decisions but also offers specific, quantifiable maintenance programs, marking a stride towards more environmentally friendly and cost-effective road maintenance. It also contributes to the sustainability of pavement maintenance.

GoogleNet transfer learning with improved gorilla optimized kernel extreme learning machine for accurate detection of asphalt pavement cracks

Asphalt cracks are the initial indication and major pavement structural distress in the field of civil engineering because it might potentially intimidate road traffic and highway safety. Managing and maintaining pavement structures are the two crucial issues faced eventually by road administrators. It is essential to take immediate action regarding pavement structural damages to prevent its severity. In recent times, there established numerous deep learning-based image processing methods to detect pavement cracks, but due to the presence of interference factors such as noises and road markings in the images, those methods failed to provide high accuracy and efficiency. Therefore, with the aim to yield high accuracy of crack detection, this article designs a novel asphalt pavement crack identification system “GoogleNet transfer learning with improved gorilla optimized kernel extreme learning machine” (GNet TL with IGT-KELM). The road crack images used for perusal are acquired from NHA12D dataset, in which it consists of 40 asphalt pavement images with two different viewpoints. For the purpose of data balancing, some non-crack images are gathered by field survey. The images with large noise distortions and unwanted background information influence crack detection performance so that the preprocessing steps are executed before applying it to the prediction system. The most significant crack and non-crack features from the images are extracted using GNet TL model. With the extracted features, the KELM is well-trained to detect cracks and non-cracks separately. To increase crack detection performance of the classifier, an improved gorilla troop optimizer is introduced to optimize the KELM parameters. The experimental finding reveals that the proposed detection mechanism achieves crack recognition accuracy of about 98.93%, which is considerably greater than compared baseline approaches.

Multi-stage generative adversarial networks for generating pavement crack images

The application of machine learning techniques in pavement health monitoring based on computer vision has greatly improved the accuracy and efficiency in the detection of pavement distress levels and categories. However, a persistent challenge in this field is the issue of sample imbalance, primarily arising from the scarcity of cracked pavement images, which hampers their effectiveness in road maintenance engineering. To address this issue and enhance the fast and stable generation of high-quality crack images for engineering purposes, this study proposes two frameworks based on Generative Adversarial Networks (GAN): Multi-Stage GAN-v1 and Multi-Stage GAN-v2. These frameworks break down the complex task of directly generating high-quality images into a series of incremental steps, gradually increasing the image resolution from initially generated low-precision images. Both versions, v1 and v2, consist of multiple sequentially connected generation units, with each unit utilizing the Wasserstein Generative Adversarial Network-Gradient Penalty (WGAN-GP). Furthermore, v2 has the additional capability of generating pavement crack images of specified types and simultaneously providing crack segmentation labels. This feature significantly enhances the practical applicability of the generated data in engineering contexts. In a comprehensive case study, the evaluation results clearly illustrate the superior image generation quality from the two proposed frameworks. Moreover, the results from ablation experiments, involving the training of nine state-of-the-art crack semantic segmentation and object detection networks using both generated images and real images, demonstrate the effective utility of these generated images for training pavement distress detection networks.

Using Deep Learning-Based Defect Detection and 3D Quantitative Assessment for Steel Deck Pavement Maintenance

Efficient distress detection and identification analysis are of great significances for long-span steel bridge deck pavement maintenance. This paper proposes a pavement distress recognition method based on road quality detection equipment and deep learning. Firstly, the sub-block image is used as the processing unit, and the three-dimensional image is divided into fracture surface element and background surface element. A pavement crack recognition network (PDRNet) based on convolutional neural network is proposed, which is used for automatic recognition of pavement background element and pavement crack element. Then, considering the pixel-level neighborhood characteristics of cracks in 3D pavement images, the PDRNet model is used to detect and quantitatively evaluate pavement cracks, which combines the crack elevation detection method to extract the complete contour of cracks in the crack surface element. Finally, the distress database containing three-dimensional markers is integrated into the pavement maintenance management platform. Results show that the PDRNet proposed in this paper has high prediction accuracy on the verification set and shows good robustness to various distress, and the average accuracy is above 88%. The established spatial-temporal integration platform overcomes the communication bottleneck between different types of data, and is important to improve the maintenance management level and predict the long-term development of distress.

Unmanned Aerial Vehicle (UAV)-Based Pavement Image Stitching Without Occlusion, Crack Semantic Segmentation, and Quantification

Unmanned Aerial Vehicle (UAV)-Based Pavement Image Stitching Without Occlusion, Crack Semantic Segmentation, and Quantification

The Classification of Asphalt Pavement Crack Images Based on Beamlet Transform

Pavement cracking is a common road infrastructure issue which significantly affects road performance, safety and longevity. This article employed a Beamlet Transform algorithm to detect and classify different types of flexible asphalt concrete pavement cracks. Additionally, a dedicated crack segmentation network was employed for precise segmentation of pavement crack. This approach incorporates advancements that has improve precision in crack classification and segmentation. Based on the results of the beamlet transform, significant improvements in the gray scale representation of crack, enhanced crack detection, reduced noise in crack images and a more precise measurement of cracks length were achieved. Computations were performed to determine the length of linear cracks and the area of block cracks. A total of 1000 pavement images were used for training and testing the accuracy of asphalt pavement crack detection and classification models. The research results showed that block cracking, alligator cracking, transverse cracking, and longitudinal cracking can all be recognized with a remarkable accuracy. Alligator cracks and block cracks achieved detection rates more than 90%, while detection rates for the longitudinal and transverse cracks reached more than 95% accuracy.

Aggregation segregation generative adversarial network (AG-GAN) facilitated multi-scale segregation detection in asphalt pavement paving stage

Pavement segregation usually cause engineering quality problems such as early damage of asphalt pavement. Therefore, it is of great significance to carry out segregation detection in the construction process. However, existing image detection technology has two problems: 1) the segregation boundary and the index are not clear; 2) the segregation assessment is localized. In order to find useful solutions to these two problems, this study proposed an Aggregation Segregation Generative Adversarial Network (AG-GAN) to generate virtual pavement images with segregation failure. Moreover, a multi-scale fusion method of Recurrent Neural Network (RNN) and the Global Averaging Sliding Window (GASW) algorithm is developed to achieve globalized aggregation evaluations. Based on extensive experiments tests, conclusions obtained that the accuracy rate of re-current neural network for segregation determination of pavement sequence can be up to 84.62%. In addition, automatically assessment of the segregation condition of pavement section can be achieved, which proves the practical application effectiveness of the proposed algorithm in the actual paving process.

A three-stage pavement image crack detection framework with positive sample augmentation

Most pavement crack detection methods based on deep learning rely too much on pixel-wise labels, and are facing with sample imbalance problem. This paper proposes a three-stage pavement crack detection framework with positive sample augmentation by an autoencoder-deep convolutional generative adversarial network (E-DCGAN). Positive samples are the images labeled as crack in the training datasets. Firstly, crack regions are located using graph based on visual saliency (GBVS). Secondly, a convolutional neural network (CNN) model is designed to recognize crack pixels in the located crack regions. The E-DCGAN is explored to generate crack images, which are then added as augmented positive samples in the training dataset of the CNN model. The trained CNN model takes the located crack regions as inputs, outputs the recognized crack pixels. Finally, fine crack detection is realized through region growing. Crack contours are completely depicted pixel-wise based on the recognized crack pixels and the Breadth-first search. Experiments are carried out on two benchmark datasets and in practical applications. The results showed that the framework could detect cracks at the pixel level. Its performance is improved compared with standard fully convolutional network (FCN)-based and U-shaped CNN (U-Net)-based crack detection methods. In addition, the standard FCN-based and U-Net-based methods require accurate pixel-wise annotations, while the proposed framework only requires image-wise annotations. It can be concluded that the proposed framework not only effectively avoids the dependence on pixel-level annotations, improves the sample imbalance issues, but also achieves competitive detection performance over the FCN and U-Net based methods.

Fine-grained damage detection of cement concrete pavement based on UAV remote sensing image segmentation and stitching

A hybrid model for pavement damage segmentation, stitching, and detection based on unmanned aerial vehicle (UAV) remote-sensing pavement images was proposed as high-resolution panoramic images are required to analyze pavement damage feature parameters during the inspection process. Based on target recognition, active stereo vision was used to determine the autonomy awareness of the UAV system in the 3D structure. Then, incremental search strategy was employed to triangulate scale-invariant feature transform (SIFT) point features based on geometric constraints. Perspective-n-point and random sample consensus (PNP-RANSAC) algorithm were combined to find the best matching surface for mis-matching rejection, and the trigonometric function theory was used to fuse images. Subsequently, the semantic segmentation framework based on deep learning context encoder network (CE-Net) was employed to train suitable models to simultaneously detect and segment pavement damages. Finally, the stitching technique and hue, saturation, value (HSV) segmentation technique were combined to obtain the feature parameters to be detected. This method achieved fast and accurate stitching, feature parameter segmentation, and pavement health condition assessment of long-threaded pavement disease image datasets. The splicing accuracy rate and the global feature segmentation overlap rate were greater than 84% and 80%, respectively. The experimental data show that the method is able to detect full-width pavement breakage parameters better than existing optical sensor-based automotive or drone inspection methods.

Detection Method of Cracks in Expressway Asphalt Pavement Based on Digital Image Processing Technology

Considering the limitations of the current pavement crack damage detection methods, this study proposes a method based on digital image processing technology for detecting highway asphalt pavement crack damage. Firstly, a non-subsampled contourlet transform is used to enhance the image of highway asphalt pavement. Secondly, the non-crack regions in the image are screened, and the crack extraction is completed by obtaining and enhancing the crack intensity map. Finally, the features of cracks are extracted and input into the support vector machine for classification and recognition to complete the detection of cracks in highway asphalt pavement. The experimental results show that the proposed method can effectively enhance the quality of a pavement image and precisely extract a crack area from the image with a high level of damage detection accuracy.

Aplikasi Visualisasi Pembangunan Jalan Baru Menggunakan Augmented Reality

Augmented Reality Technology that combines the real world with the virtual world, is interactive in real time, and is in the form of 3D animation. The Markerless Augmented Reality method is also called the markerless method, with this method the user no longer needs to use a marker to display digital elements. Currently, the construction of a road still uses a lot of rough drawings or designs using paper prints. The problem faced by workers is that drawings for making new roads use design drawings in 2D which are printed on paper, paper is prone to damage if it is always taken to the work site, such as being exposed to water, dust and folding which could damage the image on the printed paper. The research that will be carried out uses Augmented Reality technology which can provide information in the form of 3D road pavement images as well as information about the layers that make up the road pavement.

Multi-scale context feature and cross-attention network-enabled system and software-based for pavement crack detection

Pavement crack detection continues to be a stubborn problem given the interference of various factors in the actual pavement and the complex topological structure of asphalt pavement. Among all the obstacles, the bottleneck of pavement crack detection lies in the difficulty of segmenting the cracks in the pavement images whose edges are blurred. This paper proposes a multi-scale context feature and cross-attention based on convolutional neural network for accurate and robust pavement crack segmentation. The multi-scale context feature module is built in different deep networks to extract rich crack feature information. Subsequently, in order to effectively promote the seamless integration of features at different levels, we deploy cross-attention modules to each branch. After that, we add deep supervision to each branch to accelerate training. Finally, we integrate the outputs of each branch to obtain the final output diagram. The comparative experiments on various pavement datasets show that the method has better robustness. At the same time, this paper designs a complete system of pavement crack detection (PCD) and develops corresponding engineering application software. The PCD system can record the real-time pavement image data to the edge server, and the client can also monitor the real-time pavement images from the edge server through the HTTP protocol.