Intelligent Pavement Research Articles

A novel model for the pavement distress segmentation based on multi-level attention DeepLabV3+

The identification and recognition of pavement distress are vital for automatic pavement evaluation. The computation efficiency and accuracy are the two factors that determine the evaluation model. Our study utilizes the state-of-the-art image segmentation method of DeepLabV3+ with the attention mechanism. This study aims to conduct the pavement distress segmentation on the Crack500 and GAPs384 datasets. Critical results are comprehensively provided with a comparison of different backbones and architectures. In this study, an adaptive probabilistic sampling method is proposed and adopted to compare with the random crop and resized images. The test result of the adaptive prob-sampling method on DeepLabV3-attention architecture outperforms other model results on the Crack500 dataset. On the other way, the test result of the adaptive prob-sampling method on DeepLabV3 without attention architecture works better in the GAPs384 dataset. The different results relied on the diverse characteristics of datasets, since GAPs384 datasets have different asphalt surface types and a wide variety of distress classes rather than simple crack information which Crack500 included. In addition, some performance improvement methods, such as batch normalization of image input, revising dice loss, and hyperparameters search, have been implemented in this work. The results are solid and reliable for concluding the critical analysis of methods and datasets. This study demonstrates the considerable potential of deep learning in the application of intelligent pavement evaluation. To advance the practical application of artificial intelligence on distress segmentation, we will explore more domain adaptive methods in future studies.

Automated instance segmentation of asphalt pavement patches based on deep learning

The location and pixel-level information of the patch are all critical data for the quantitative evaluation of pavement conditions. However, obtaining both parch location and pixel-level information simultaneously is a challenge in intelligent pavement patch surveys. This paper proposes a deep-learning-based patch instance segmentation network (PISNet) that employs you only look once (YOLO)v5 as the baseline and adds a semantic segmentation branch to provide an effective solution for this challenge. The proposed PISNet replaces the original backbone CSPDarknet53 and neck of YOLOv5 with a novel feature extractor named symmetrical pyramid network (SPN). The proposed SPN aims at repeating fusion and transfer of shallow semantic features and deep spatial localization features in the order of “FPN-PAN-FPN” such that the multi-scale semantic expression and localization ability of the feature map could be enhanced. Moreover, a modified feature selection module is also incorporated into the SPN as a skip connection to aggregate more spatial details of the feature map while suppressing redundant features. Experimental results show that compared with Mask region convolutional neural network (R-CNN), You only look at coefficients (YOLACT), YOLACT++, EfficientDet, Fully convolutional one stage object detector (FCOS), You only look once version 5m (YOLOv5m), U-Net, DeepLabv3+, and High resolution network-object contextual representations (HRNet-OCR), the proposed PISNet has the best detection performance. Meanwhile, the proposed PISNet achieves superior accuracy/frames per second trade-offs compared to Mask R-CNN, YOLACT, and YOLACT++. Particularly, the proposed PISNet has certain promising potential in supporting pavement patch detection in real-time scenarios and potentially degraded pavement patch detection. Moreover, the proposed PISNet can yield superior segmentation results compared with Mask R-CNN, YOLACT, YOLACT++, U-Net, HRNet-OCR, and DeepLabv3+ on public CRACK500 datasets. Code has been made available at: https://github.com/716HAZ/PISNet .

A new highway cost allocation framework in the day of connected and autonomous vehicles

The objectives of the present study are to explore potential changes expected in highway infrastructure expenditures and revenues, and develop a robust framework that will facilitate highway cost allocation when connected and autonomous vehicles (CAVs) are in operation. The scope of this study covered both federal and state expenditures and revenue sources. The highway cost allocation elements covered in the paper include vehicle miles of travel, highway expenditures highway revenues, and equity analysis. In the day of CAVs, significant changes are expected in all these elements, and these changes are incorporated into the proposed highway cost allocation framework. Pavement-related expenditures would increase due to the addition of CAV-related infrastructure integrated into the pavement infrastructure to ensure it is an intelligent pavement. Bridge-related expenditures are expected to increase due to the infrastructure technologies on the bridges making them smart bridges, and the design loads of these bridges may have to be increased due to potential platooning on long span bridges. It is expected that many of the CAVs on the road will be electric vehicles. All these changes are incorporated into the proposed highway cost allocation framework.

Asphalt Pavement Damage Detection through Deep Learning Technique and Cost-Effective Equipment: A Case Study in Urban Roads Crossed by Tramway Lines

Asphalt pavements are subject to regular inspection and maintenance activities over time. Many techniques have been suggested to evaluate pavement surface conditions, but most of these are either labour-intensive tasks or require costly instruments. This article describes a robust intelligent pavement distress inspection system that uses cost-effective equipment and the ‘you only look once’ detection algorithm (YOLOv3). A dataset for flexible pavement distress detection with around 13,135 images and 30,989 bounding boxes of damage was used during the neural network training, calibration, and validation phases. During the testing phase, the model achieved a mean average precision of up to 80%, depending on the type of pavement distress. The performance metrics (loss, precision, recall, and RMSE) that were applied to estimate the object detection accuracy demonstrate that the technique can distinguish between different types of asphalt pavement damage with remarkable accuracy and precision. Moreover, the confusion matrix obtained in the validation process shows a distress classification sensitivity of up to 98.7%. The suggested technique was successfully implemented in an inspection car. Measurements conducted on urban roads crossed by tramway lines in the city of Palermo proved the real-time ability and great efficacy of the detection system, with potentially remarkable advances in asphalt pavement examination efficacy due to the high rates of correct distress detection.

Intelligent Pavement Diseases Detection Model based on YOLOv5s

There are many kinds of pavement diseases, and the detection background in the actual scene is complex, so the accuracy of the detection model is not high. Based on YOLOv5s, a pavement diseases detection model YOLOv5s-CBP is proposed, and the CBAM (Convolutional Block Attention Module) attention module was introduced into the backbone network to enhance the model's attention to the characteristics of pavement diseases. The BiFPN network was constructed in the neck, which realizes the cross-layer exchange and integration of information and retains more high-level semantic information. A small target detection layer P2 was added to the detection head to improve the detection ability of the model for small road diseases. The experimental results on data sets show that the precision and recall of the model are improved by 6.1% and 4.4% respectively, and the mAP is improved by 5.7%, which effectively improves the performance of the detection model.

Real-time instance-level detection of asphalt pavement distress combining space-to-depth (SPD) YOLO and omni-scale network (OSNet)

Pavements function as protective layers for roads and require frequent inspection and maintenance throughout their service life. This paper describes an intelligent pavement distress inspection system that uses an enhanced version of the ‘you only look once’ (YOLO) model and an omni-scale network (OSNet) to instantly capture road surface distress images and their precise locations. The YOLO model was evaluated on a dataset comprising 9749 pavement distress images, with the detected distress serving as an input for feature extraction and instance-level recognition through OSNet. The OSNet model achieved a mean average precision (mAP) of 99.4% for a dataset containing 398 individual distress instances. The proposed methods were successfully integrated into a pavement distress inspection vehicle. Field experiments demonstrated the real-time capability and high efficiency of the system, with significant improvement in road maintenance inspection efficiency

Automated crack segmentation on 3D asphalt surfaces with richer attention and hybrid pyramid structures

ABSTRACT Automated pavement crack detection is crucial to supporting fine pavement maintenance and ensuring safety for road facilities. Due to the complex pavement condition and crack features, it is still a critical challenge in intelligent pavement surveys. This paper proposed a novel pixel-level pavement crack segmentation network, PCSNet, to provide a solution to this challenge. The network has richer attention and hybrid pyramid structures, which implement full-process crack feature fusion and enhancement. The richer attention module consists of cascaded self-attention and attention gate modules. It captures the crack spatial dependence information and prunes the feature response. The hybrid pyramid structures consist of a multistage convolutional pyramid module and a pyramid pooling module. It integrates contextual information at multiple receptive field scales to enhance the potential crack feature representation. The proposed structure enriches the crack details and optimises the scene parsing on the global geometry of the cracks. A sizeable 3D pavement crack dataset is built for training and testing. The proposed network exhibited the best performance, achieving F1-score, mean intersection of union, and mean pixel accuracy of 81.21%, 77.13%, and 87.17%, respectively. The network can reconstruct the complete crack geometry, preserve the crack edges well, and optimises the detection of shallow and complex cracks. The method exhibits superior and robust performance, facilitating accurate pavement technical condition assessment and maintenance decisions.

Textile-inspired triboelectric nanogenerator as intelligent pavement energy harvester and self-powered skid resistance sensor

Triboelectric nanogenerator (TENG) provides a new idea for harvesting low-frequency energy tire-road interaction and self-powered sensors. A real-time assessment of skid resistance provides valuable information to deliver appropriate driving manner to ensure safety in all weather conditions. However, there is no mature perception technology in the previous study due to the complexity of tire-road interaction. Inventing appropriate devices and solutions for harvesting wasted energy and obtaining skid resistance information in real-time is essential. Here, a textile-inspired TENG is developed, which was integrated into the tire cord for energy harvesting and self-powered skid resistance monitoring. As the theoretical basis, the capability of the proposed textile-inspired TENG for energy harvesting and skid-resistance monitoring was demonstrated based on the coupling of vehicle-road dynamics and the electrodynamics. Consequently, the average power density reaches to 4.219 mW/m2(output power with 82.7 μW). Besides, the short-circuit current waveform of the intelligent tires was firstly applied to distinguish the difference of the pavement skid resistance caused by the roughness characteristics of the pavement structure. The open-circuit voltage amplitude was adopted to evaluate the correlation quantitatively between pavement skid resistance and the TENG's output signal, and the accuracy of this system can reach 96%. This work innovatively proposes a new idea for simultaneously harvesting wasted energy from tire-road interaction and improving the level of traffic safety and expands the application of triboelectric nanogenerators for automatic driving technology.

Deep learning framework for intelligent pavement condition rating: A direct classification approach for regional and local roads

Transport authorities rely on pavement characteristics to determine a pavement condition rating index. However, manually computing ratings can be a tedious, subjective, time-consuming, and training-intensive process. This paper presents a deep-learning framework for automatically rating the condition of rural road pavements using digital images captured from a dashboard-mounted camera. The framework includes pavement segmentation, data cleaning, image cropping and resizing, and pavement condition rating classification. A dataset of images, captured from diverse roads in Ireland and rated by two expert raters using the pavement surface condition index (PSCI) scale, was created. Deep-learning models were developed to perform pavement segmentation and condition rating classification. The automated PSCI rating achieved an average Cohen Kappa score and F1-score of 0.9 and 0.85, respectively, across 1–10 rating classes on an independent test set. The incorporation of unique image augmentation during training enabled the models to exhibit increased robustness against variations in background and clutter.

Automated pothole condition assessment in pavement using photogrammetry-assisted convolutional neural network

ABSTRACT Automated detection of pavement distress can prevent deterioration of premature surface disintegration in pavements. Potholes that are a common sight in harsh and cold terrains are a severe threat to road safety and a major contributing factor to pavement distress. To facilitate timely detection and repair of potholes, a computationally light and feasible, intelligent pavement pothole detection system is proposed by developing a novel workflow for image-based detection and severity assessment. A single-stage CNN architecture, RetinaNet is modified and optimised to best detect potholes and used in combination with a novel pothole depth estimation algorithm. A comparative evaluation of the model’s performance against the existing state-of-the-art model on the benchmark dataset establishes the proposed model’s high performance and applicability in real-time scenarios. The depth estimation algorithm is based on a 3D road surface model generated by employing the photogrammetric process of structure from motion (SfM). The point cloud data obtained thereafter, is used for accurate measurement of pothole depth. The comparison of the derived depth with the onsite depth measurement of the pothole reveals a mean error below 5%. This method leads to a practical and intelligent solution to be implemented as part of a potential pavement health assessment system for future practice.

Research on Feature Identification and Trajectory Planning of Pavement Cracks

As the most common method to detect pavement cracks, manual detection has uncontrollable factors such as low efficiency, inconsistent standards and easy to be interfered with by external forces, so it is not suitable for pavement crack detection in today’s intricate traffic network. In order to improve the efficiency of pavement repair and reduce the labor cost of the repair process, this paper proposes an intelligent pavement crack detection and repair algorithm. The algorithm uses image numerical parameters to classify cracks with different geometric features and extracts texture geometric features of various types of cracks based on different filtering strategies. It solves the problem that traditional single filtering algorithms are difficult to extract features according to the different characteristics of the collected image, which leads to the loss of information. Finally, the algorithm establishes a mathematical model for efficient trajectory planning combined with the nozzle size of the crack-repairing machine. In this paper, the robustness and efficiency test of the algorithm is carried out on the pavement image dataset with various types of cracks, and the experiment is carried out on the intelligent pavement crack detection and repair prototype, which verifies the accuracy and reliability of the planned trajectory.

Prediction of the skid-resistance deterioration in asphalt pavement based on peephole–LSTM neural network

ABSTRACT Poor pavement skid resistance contributes to frequent traffic accidents, with the asphalt surface rapidly deteriorating over time. Accurate prediction of skid resistance behavior is essential for road safety. This study introduces a pavement accelerated polishing system, collecting data from 11 stages using a British pendulum tester and a high-precision three-dimensional laser scanner. It develops a predictive model, FrictionNet-II, based on a peephole long-short time memory (peephole-LSTM) neural network. The model uses data from the first eight polishing stages as input and the last three as output, allowing for predictions of pavement skid resistance after 160,000, 180,000, and 200,000 cycles based on the friction deterioration curve of the initial stages (0 to 140,000 cycles). FrictionNet-II accurately forecasts skid resistance after 160,000, 180,000, and 200,000 cycles, achieving R2 values of 88%, 90%, and 90%, respectively. This research demonstrates the potential of deep learning in predicting asphalt pavement skid resistance, laying the foundation for intelligent pavement preservation and improved skid-resistance performance.

Designing and Building an Intelligent Pavement Management System for Urban Road Networks

Pavement maintenance plays a significant role in megacities. Managing complaints and scheduling road reviews are the two maintenance concerns under the intelligent pavement management system (PMS) plan. In contrast, if the damages are not treated immediately, they will increase over time. By leveraging accurate data from sensors, smart PMS will improve management capability, support sustainability, and drive economic growth in the road network. This research aimed to elaborate on the different modules of an intelligent city pavement network to advance to a sustainable city. First, a 3D mobile light detection and ranging (LiDAR) sensor, accompanied by a camera, was applied as the data collection tool. Although 3D mobile LiDAR data have gained popularity, they lack precise detection of pavement distresses, including cracks. As a result, utilizing RGB imaging may help to detect distresses properly. Two approaches were integrated alongside conducting the data analysis in this paper: (1) ArcGIS pro, developed by Esri Inc., which includes noise removal, digital elevation model (DEM) generation, and pavement and building footprint extraction; (2) the Mechanistic-Empirical Pavement Design Guide (AASHTOWare PMED), which was used to assess site specifications such as traffic, weather, subbase, and current pavement conditions in an effort to design the most appropriate pavement for each road section. For the 3D visualization module, CityEngine (a software from Esri) was used to provide the 3D city model. After implementing the research methodology, we drew the following conclusions: (1) using the AASHTOWare PMED method to make decisions about road maintenance and rehabilitation(M&R) actions can significantly speed up the decision-making process, essentially saving time and money and shortening the project’s duration; and (2) if the road conditions are similar, the smart geographical information system (GIS)-based PMS can make consistent decisions about road M&R strategies, i.e., the interference from human factors is less significant.

Super-Resolution Reconstruction Method of Pavement Crack Images Based on an Improved Generative Adversarial Network.

A super-resolution reconstruction approach based on an improved generative adversarial network is presented to overcome the huge disparities in image quality due to variable equipment and illumination conditions in the image-collecting stage of intelligent pavement detection. The nonlinear network of the generator is first improved, and the Residual Dense Block (RDB) is created to serve as Batch Normalization (BN). The Attention Module is then formed by combining the RDB, Gated Recurrent Unit (GRU), and Conv Layer. Finally, a loss function based on the L1 norm is utilized to replace the original loss function. The experimental findings demonstrate that the self-built pavement crack dataset's Peak Signal-to-Noise Ratio (PSNR) and Structural Similarity (SSIM) of the reconstructed images reach 29.21 dB and 0.854, respectively. The results improved compared to the Set5, Set14, and BSD100 datasets. Additionally, by employing Faster-RCNN and a Fully Convolutional Network (FCN), the effects of image reconstruction on detection and segmentation are confirmed. The findings indicate that the segmentation results' F1 is enhanced by 0.012 to 0.737 and the detection results' confidence is increased by 0.031 to 0.9102 when compared to state-of-the-art methods. It has a significant engineering application value and can successfully increase pavement crack-detecting accuracy.

Intelligent pixel‐level detection of multiple distresses and surface design features on asphalt pavements

AbstractSimultaneous pixel‐level detection of multiple distresses and surface design features on complex asphalt pavements is a critical challenge in intelligent pavement survey. This paper proposes a deep‐learning model named ShuttleNet to provide an efficient solution for this challenge by implementing robust semantic segmentation on asphalt pavements. The proposed ShuttleNet aims at repeating the encoding–decoding round freely or even endlessly such that the contexts at different resolution levels can be learned and integrated many times for enhanced latent representations. Additionally, a new and efficient connection method called memory connection is also proposed in the paper and deployed in the ShuttleNet model to provide shortcut connections between successive encoding–decoding rounds. The proposed memory connection can partially or entirely carry the decoded information at different resolution levels into the next encoding–decoding round. Pairing 3D pavement images with 2D pavement images, the proposed ShuttleNet model is applied to detect multiple distresses and surface design features on asphalt pavements simultaneously, including pavement cracks, potholes, sealed cracks, patches, markings, expansion joints, and the pavement background. Experimental results demonstrate that the mean F‐measure and mean intersection‐over‐union attained by the recommended architectural variation of the proposed ShuttleNet model on 1500 testing image pairs are 92.54% and 0.8657 respectively. According to the performance comparisons using both private and public datasets, the proposed ShuttleNet model can yield a noticeably higher detection accuracy, compared with four state‐of‐the‐art models for semantic segmentation.

Evaluation of Pavement Roughness by the International Roughness Index for Sustainable Pavement Construction in New Taipei City

Good road quality is an indicator of urban progress. In August 1998, the New Taipei City government implemented the “Road Leveling Project” on roads within its jurisdiction to improve the quality of public transportation. This paper aimed to analyze the International Roughness Index (IRI) of 152 asphalt concrete pavement sections in New Taipei City before and after road leveling. An intelligent pavement inspection vehicle that met the requirements of E 950 Class I was used to detect the IRI values of the road sections before and after maintenance. The statistical analysis showed that the implementation of the Road Leveling Project had a significant effect on improving the road roughness of roads under its jurisdiction. Moreover, it became possible to arrange a budget for roads with IRI values above a threshold or to estimate the budget needed to improve roads with an IRI higher than a certain value.

New innovations in pavement materials and engineering: A review on pavement engineering research 2021

Sustainable and resilient pavement infrastructure is critical for current economic and environmental challenges. In the past 10 years, the pavement infrastructure strongly supports the rapid development of the global social economy. New theories, new methods, new technologies and new materials related to pavement engineering are emerging. Deterioration of pavement infrastructure is a typical multi-physics problem. Because of actual coupled behaviors of traffic and environmental conditions, predictions of pavement service life become more and more complicated and require a deep knowledge of pavement material analysis. In order to summarize the current and determine the future research of pavement engineering, Journal of Traffic and Transportation Engineering (English Edition) has launched a review paper on the topic of “New innovations in pavement materials and engineering: A review on pavement engineering research 2021”. Based on the joint-effort of 43 scholars from 24 well-known universities in highway engineering, this review paper systematically analyzes the research status and future development direction of 5 major fields of pavement engineering in the world. The content includes asphalt binder performance and modeling, mixture performance and modeling of pavement materials, multi-scale mechanics, green and sustainable pavement, and intelligent pavement. Overall, this review paper is able to provide references and insights for researchers and engineers in the field of pavement engineering.

An improved minimal path selection approach with new strategies for pavement crack segmentation

Intelligent pavement detection technology provides a new research idea for identifying pavement cracks. To solve the problem that traditional minimal cost path selection (MPS) is computationally intensive and inefficient in crack segmentation, this study proposes an improved minimal cost path algorithm (IMPS). The contribution of IMPS is reflected in its more efficient path planning than MPS and better choice of source and destination of each path by using new strategies. First, a grayscale pavement image is divided into nonoverlapping cells with a certain size. Next, the darkest points in each cell are selected as candidate points and adjacent candidate points are connected based on IMPS. Then pseudocracks are removed based on postprocessing steps before cracks are well segmented. IMPS is compared with other crack segmentation algorithms to prove its reliability. The efficiency of IMPS is increased by approximately 56% than MPS. The accuracy of IMPS is higher than that of MPS and other methods, with Mean Intersection over Union reaching 73.99%.

Development and Prospect of Intelligent Pavement

Development and Prospect of Intelligent Pavement

Intelligent Pavement Damage Monitoring Research in China

The development of artificial intelligence technology is of vital significance to the progress of human civilization. Through the development of this aspect in recent years, China has achieved unprecedented gains in equipment and software technology, such as road damage monitoring. However, it is difficult to deal with a large amount of data and real-time data. In addition, China is the third largest country in the world in terms of land area, with well-developed roads and more and more road construction, which increases the workload of road damage monitoring. In order to solve these problems, a research scheme of intelligent pavement damage monitoring is proposed. The main idea of the scheme is to obtain point cloud data by scanning the road surface with unmanned control equipment, carry out Harris, star and sift detection on the pictorial point cloud data, and then judge the damaged area by combining with the characteristic matrix of the damaged data. Through the processing of point cloud data obtained by the existing manned control equipment, it is found that it has a good effect on the monitoring of pavement crack damage. From the actual results obtained, it can be judged that the pavement damage of single crack and multi-crack can be determined. Intelligent development is an application that integrates multiple interdisciplinary subjects. This research can provide some development ideas for intelligent road monitoring, and at the same time, it is hoped to contribute to the development and integration of interdisciplinary subjects.