Pavement Images Research Articles

Research on Pavement Crack Detection Algorithm based on Deep Residual Unet Neural Network

Pavement crack is one of the main factors threatening highway safety. Rapid and accurate identification of pavement crack is very important to protect highway. Traditional image processing methods have been difficult to meet the requirements of practical engineering. So, this paper proposes an improved pavement crack detection method based on deep convolution neural network model, to overcome the inaccurate segmentation caused by complex background and fuzzy crack edge in asphalt pavement image, which is an encoder-decoder network structure, and integrates the idea of semantic segmentation into image detection. The encoder of the proposed method adopts the ResNet’s residual structure to strengthen image feature extraction, the decoder uses transpose convolution from the original UNet structure for up sampling, and integrates the encoder features with the up sampled features after cutting, Strengthen the learning of network features. Experimental results show that the F1 score, Precision and Recall are significantly improved on the asphalt crack dataset. Compared with the traditional algorithm, the proposed algorithm can segment the cracks better and improve the accuracy of pavement crack detection.

A Fast Inference Vision Transformer for Automatic Pavement Image Classification and Its Visual Interpretation Method

Traditional automatic pavement distress detection methods using convolutional neural networks (CNNs) require a great deal of time and resources for computing and are poor in terms of interpretability. Therefore, inspired by the successful application of Transformer architecture in natural language processing (NLP) tasks, a novel Transformer method called LeViT was introduced for automatic asphalt pavement image classification. LeViT consists of convolutional layers, transformer stages where Multi-layer Perception (MLP) and multi-head self-attention blocks alternate using the residual connection, and two classifier heads. To conduct the proposed methods, three different sources of pavement image datasets and pre-trained weights based on ImageNet were attained. The performance of the proposed model was compared with six state-of-the-art (SOTA) deep learning models. All of them were trained based on transfer learning strategy. Compared to the tested SOTA methods, LeViT has less than 1/8 of the parameters of the original Vision Transformer (ViT) and 1/2 of ResNet and InceptionNet. Experimental results show that after training for 100 epochs with a 16 batch-size, the proposed method acquired 91.56% accuracy, 91.72% precision, 91.56% recall, and 91.45% F1-score in the Chinese asphalt pavement dataset and 99.17% accuracy, 99.19% precision, 99.17% recall, and 99.17% F1-score in the German asphalt pavement dataset, which is the best performance among all the tested SOTA models. Moreover, it shows superiority in inference speed (86 ms/step), which is approximately 25% of the original ViT method and 80% of some prevailing CNN-based models, including DenseNet, VGG, and ResNet. Overall, the proposed method can achieve competitive performance with fewer computation costs. In addition, a visualization method combining Grad-CAM and Attention Rollout was proposed to analyze the classification results and explore what has been learned in every MLP and attention block of LeViT, which improved the interpretability of the proposed pavement image classification model.

Crack image recognition on fracture mechanics cross valley edge detection by fractional differential with multi-scale analysis

The recognition of pavement cracks is crucial in road engineering and airport maintains. In order to successfully apply image processing technique for automatic crack detection, the first and hardest task is to recognize crack images in a huge number of pavement images. To do this, the image processing technique and Fracture mechanics are combined first time in this area, the studied method includes four steps: (1) The pavement crack image shrinking is carried out by a proposed multi-scale analysis algorithm, which is more effective for both preserving weak valley edges and reducing computing cost; (2) Then, a so called valley edge detection algorithm based on Fractional differential for finding local dark line/curve is studied for tracing crack segments, it considers template size, weighted average gray level value in each line in four different directions, the output can be a gradient magnitude image or a binary image; (3) In the binary image, the crack segments are refined based on a number of post processing functions to remove noise and fill segment gaps; and (4) After that, to quickly judge if the image has cracks, Fracture mechanics is applied to calculate the judgment parameter T, which is directly proportion to the image edge density, and the ratio between the average gradient magnitude value and the average gray level value in the candidate crack segment. In experiments, more than 400 pavement images (the resolution is 4096 × 2048 pixels) are tested, and the crack identification accuracy is up to 97%.

Foreign Object Debris Detection for Optical Imaging Sensors Based on Random Forest.

In recent years, aviation security has become an important area of concern as foreign object debris (FOD) on the airport pavement has a huge potential risk to aircraft during takeoff and landing. Therefore, accurate detection of FOD is important to ensure aircraft flight safety. This paper proposes a novel method to detect FOD based on random forest. The complexity of information in airfield pavement images and the variability of FOD make FOD features difficult to design manually. To overcome this challenge, this study designs the pixel visual feature (PVF), in which weight and receptive field are determined through learning to obtain the optimal PVF. Then, the framework of random forest employing the optimal PVF to segment FOD is proposed. The effectiveness of the proposed method is demonstrated on the FOD dataset. The results show that compared with the original random forest and the deep learning method of Deeplabv3+, the proposed method is superior in precision and recall for FOD detection. This work aims to improve the accuracy of FOD detection and provide a reference for researchers interested in FOD detection in aviation.

Asphalt Pavement Crack Detection Based on Fusion of Neighborhood and Gradient Salient Features

In order to solve the problem of the rich texture noise and serious misidentification of small cracks in the asphalt pavement crack detection,, an asphalt pavement crack detection method that combines neighborhood and gradient salient features is proposed. Firstly, the grayscale correction and morphological reconstruction are used to reduce the influence of external interference and uneven brightness caused by the bright spot noise in the rich texture, then the neighborhood salient features are extracted according to the salient differences between pixels and their neighborhood, and gradient salient features in different directions are obtained through the steerable filter. With the convolutional fusion of the neighborhood and gradient salient features, preferential directions are chosen to generate the salient map of feature fusion. Secondly, the suspected crack aggregation areas are obtained by the threshold segmentation of salient map. Based on the different geometric characteristics of the aggregation areas, a clustering analysis method is introduced to select the crack candidate area. Finally, the method of searching and locating a regional endpoint is proposed to eliminate the subsets without endpoint aggregation area, the endpoints of different regions are connected to achieve the complete crack extraction. The experimental results of the collected asphalt pavement crack image datasets show that the precision, recall and <italic>F</italic>-measure value are 92.857%, 86.405% and 89.515%, which can effectively detect cracks in asphalt pavement images, especially for small cracks, and provide more accurate crack information for pavement maintenance.

Automated pavement distress segmentation on asphalt surfaces using a deep learning network

ABSTRACT Recently, many deep learning methods have achieved great results in the field of automated pavement distress detection, but most of them ignore other types of distresses beyond cracks. This paper proposes an efficient deep learning framework for automated asphalt pavement distress segmentation called pavement distress segmentation network (PDSNet). PDSNet can effectively segment multiple asphalt pavement distresses, including crack, pothole, raveling, patch, and sealed crack. It consists of two parallel feature extraction branches. One is the P branch to extract prior global information. The other is the U branch to obtain local information. By utilising the global and local features together, PDSNet can produce precise segmentation results under complicated circumstances. For deep learning purposes, a pavement distress dataset consisting of 4000 pavement images is collected and manually labelled at pixel level. Each image of the pavement dataset is a two-channel image, which is concatenated by a 2D pavement image and the correspondingly 3D pavement image. Particularly, it is the first pavement distress dataset that utilises two-channel pavement images. According to the experimental results, PDSNet yields a performance with a MIoU of 83.7%. Compared with the state-of-the-art networks, PDSNet achieves the best MIoU and has considerable parameter number and inference time.

Convolution neural network model for an intelligent solution for crack detection in pavement images

Convolution neural network model for an intelligent solution for crack detection in pavement images

Research on Automatic Dodging Method for Road Surface Line Array Images

Line scan cameras are widely used to acquire pavement images due to their imaging characteristics and low price, but the acquired pavement line array images often have uneven brightness. To eliminate the impact of image greyscale unevenness on subsequent processing, an automatic dodging method is proposed based on mask dodging algorithm. The main idea of the method is to chuck the images into sub‐blocks with the approximate greyscale change degree by introducing the cumulative slope conception. The rules to determine the filter size and the parameters required to adjust the sub‐block are self‐adaptive. The contrast stretching method is improved to make it more suitable for pavement images. The method is compared with other classical dodging algorithms, and the results show that the proposed automatic dodging method is better than others intuitively and quantitatively. The image length has no significant influence on it. Moreover, the crack segmentation results indicate that the proposed dodging method is effective for line array images, and a more accuracy crack segmentation could be achieved if the images preprocessed by the proposed dodging methods.

Convolution neural network model for an intelligent solution for crack detection in pavement images

Convolution neural network model for an intelligent solution for crack detection in pavement images

A Novel Approach for Detection of Pavement Crack and Sealed Crack Using Image Processing and Salp Swarm Algorithm Optimized Machine Learning

During the phase of periodic survey, sealed crack and crack in asphalt pavement surface should be detected accurately. Moreover, the capability of identifying these two defects can help reduce the false‐positive rate for pavement crack detection. Because crack and sealed crack are both line‐based defects and may resemble each other in shape, this study puts forward an innovative method based on computer vision for detecting sealed crack and crack. This method is an integration of feature extraction based on image processing and metaheuristic optimized machine learning. Image processing is used to compute features that characterize visual appearance and texture of the pavement image. Subsequently, Salp Swarm Algorithm integrated with multiclass support vector machine is employed for pattern recognition. Based on experimental results, the newly developed method has achieved the most desired predictive performance with an accuracy rate = 91.33% for crack detection and 92.83% for sealed crack detection.

Transfer learning based deep convolutional neural network model for pavement crack detection from images

The road is a path that supports to connect different places. It plays a crucial role in our day-today life. Improper maintenance, overloading, climate conditions, and some other elements create distress on the roads. The common distresses are Potholes, cracking, and rutting. Manually detecting the distresses means human inspection is a messy and long time-consuming process. In recent past accidents on road is on the increase due to improper maintenance of road. Efficient methods of detecting pavement damages using image processing, machine learning and deep learning techniques have been a trending research topic. Image processing algorithms mainly include edge detection, region growing methods, and threshold segmentation operations for processing the pavement images and extracting crack information from the images. Machine learning methods of pavement crack detection adapts neural networks, supervised and unsupervised learning algorithms with pavement crack image as input. With Deep learning techniques, it has been possible to detect pavement cracks with greater accuracy. In this paper, we review the deep learning methods of pavement crack detection and propose a novel method to detect pavement cracks using Deep Learning with transfer learning. We also analyzed the performance of the proposed model for different network architectures namely, Google net, Alexnet and Resent and inferred that Google net gives better performance in detecting pavement cracks.

Asphalt Pavement Potholes Localization and Segmentation using Deep RetinaNet and Conditional Random Fields

The main aspect of maintaining the roads and highways' durability and long life is to detect potholes and restore them. A huge number of accidents occur on the roads and highways due to the pothole. It also causes financial loss to vehicle owners by damaging the wheel and flat tire. For the strategies of the road management system and ITS (Intelligent Transportation System) service, it is one of the major tasks to quickly and precisely detect the potholes. To solve this problem, we have proposed a deep learning methodology to automatically detect and segment the pothole region within the asphalt pavement images. The detection of the pothole is a challenging task because of the arbitrary shape and complex structure of the pothole. In our proposed methodology, to accurately detect the pothole region, we used RetinaNet that creates the bounding box around the multiple regions. For the segmentation we used Conditional Random Field that segments the detected pothole regions obtained from RetinaNet. There are three steps in our methodology, image preprocessing, Pothole region localization, and Pothole segmentation. Our proposed methodology results show that potholes in the images were correctly localized with the best accuracy of 93.04%. Conditional Random Fields (CRF) also show good results.

A Novel Vision Transformer Approach for Automatic Asphalt Pavement Image Classification and its Visual Interpretation Methods

A Novel Vision Transformer Approach for Automatic Asphalt Pavement Image Classification and its Visual Interpretation Methods

Evaluation of Compression Algorithms for 3D Pavement Image Data

A standard interchangeable data format was recently developed through the Transportation Pooled-Fund Study TPF-5(299) to meet different requirements and needs for two- and three-dimensional (2D/3D) pavement image data storage and exchange. Considering pavement images occupy large amounts of storage space, efficient compression algorithms for both 8-bit 2D and higher-bit 3D images become one of the key components for a successful implementation of the standard. Highway agencies and industry vendors exclusively use the Joint Photographic Experts Group (JPEG) or its successor JPEG 2000 standard for 2D image compression, while the compression of 3D 16-bit images remains to be proprietary. The goal of this paper is to evaluate suitable algorithms for compressing 3D pavement images. An overview of the state-of-the-art pavement image data collection technologies was performed, and the relevant image data formats were identified. Subsequently, five compression algorithms suitable for 16-bit image data were selected based on results from a comprehensive survey of the TPF-5(299) technical community. The compressions of several representative field images were implemented and evaluated in terms of the computation efficiency and retaining of image quality. The expected benefits of this study include facilitating standard and efficient pavement image data collection and accelerating the development of consistent pavement condition evaluation.

Pavement distress detection using convolutional neural networks with images captured via UAV

Pavement distress detection is crucial in the decision-making for maintenance planning. Unmanned aerial vehicles (UAVs) are helpful in collecting pavement images. This paper proposes the collection of pavement distress information using a UAV with a high-resolution camera. A UAV platform for pavement image collection was assembled, and the flight settings were studied for optimal image quality. The collected images were processed and annotated for model training. Three state-of-the-art object-detection algorithms—Faster R-CNN, YOLOv3, and YOLOv4, were used to train the dataset, and their prediction performances were compared. A pavement image dataset was established with six types of distress. YOLOv3 demonstrated the best performance of the three algorithms, with a mean average precision (MAP) of 56.6%. The findings of this study assist in the inspection of non-destructive automatic pavement conditions.

A novel evaluation method of aggregate distribution homogeneity for asphalt pavement based on the characteristics of texture structure

To realize the non-destructive detection of aggregate distribution homogeneity for asphalt pavement, the texture imaging device was developed to acquire the surface image of asphalt pavement. According to the distribution characteristics of texture structure, an improved segmentation algorithm for pavement texture image was proposed. A novel evaluation method of aggregate distribution homogeneity for asphalt pavement based on digital image processing (DIP) technique was discussed. The rutting specimens of asphalt mixture with standard segregation levels were simulated in laboratory, and the evaluation standard of aggregate distribution homogeneity was established. The results show that the improved threshold segmentation method combining rectangular partition and Otsu algorithm realizes the effective segmentation of texture structure from pavement image. The proposed index (D) for aggregate distribution homogeneity based on the quantity and location of texture structure has a good correlation with the texture depth, which can be used to evaluate the aggregate distribution homogeneity of asphalt pavement comprehensively. The evaluation standard of aggregate distribution homogeneity based on D value provides the reference to analyze the construction quality of asphalt pavement.

CrackLabel: A Thresholding-Based Crack Labeling Tool for Asphalt Pavement Images

In an image classification system based on deep learning, a training dataset is a set of labelled images and is often composed of a large number of images. Image labelling tool is usually used to facilitate in creating the training dataset used by the classifier during the learning phase. This paper presents a new image labelling tool called CrackLabel that can automatically label the cracks in the asphalt pavement images. A specially designed image thresholding method called the Global and Lower Quartile Average Intensity (GLQAI) method is utilised. In this study, the training dataset is developed by using real pavement images that resized to 1024×768 resolution. First, crack images are automatically segmented into 768 small patches with 32×32 resolution (pixel). Then, a threshold-based method is applied to automatically segment these patches into two classes which are crack and non-crack patches. The image thresholding method based on the average of global average intensity (GAI) and lower quartile intensity (LQI), namely GLQAI is proposed for this task. Next, the labelling process is performed by assigning patches associated with the crack and background into the crack and non-crack folder, respectively. Finally, the performance of CrackLabel is benchmarked by comparing the results with the manual label crack images by human experts, and three commonly used thresholding methods; Otsu, Kapur and Kittler-Illingworth thresholding. Experimental results show that the proposed thresholding method achieved the best classification rate among various thresholding methods with 94.50%, 93.60% 94.00% and 94.05% for recall, precision, accuracy, and F-score respectively. In conclusion, it is observed that the proposed method using the newly threshold algorithm is very effective in label images into the crack and non-crack patches to maximize the training performance.

Attention-Based Multi-Scale Convolutional Neural Network (A+MCNN) for Multi-Class Classification in Road Images.

Automated pavement distress recognition is a key step in smart infrastructure assessment. Advances in deep learning and computer vision have improved the automated recognition of pavement distresses in road surface images. This task remains challenging due to the high variation of defects in shapes and sizes, demanding a better incorporation of contextual information into deep networks. In this paper, we show that an attention-based multi-scale convolutional neural network (A+MCNN) improves the automated classification of common distress and non-distress objects in pavement images by (i) encoding contextual information through multi-scale input tiles and (ii) employing a mid-fusion approach with an attention module for heterogeneous image contexts from different input scales. A+MCNN is trained and tested with four distress classes (crack, crack seal, patch, pothole), five non-distress classes (joint, marker, manhole cover, curbing, shoulder), and two pavement classes (asphalt, concrete). A+MCNN is compared with four deep classifiers that are widely used in transportation applications and a generic CNN classifier (as the control model). The results show that A+MCNN consistently outperforms the baselines by 1∼26% on average in terms of the F-score. A comprehensive discussion is also presented regarding how these classifiers perform differently on different road objects, which has been rarely addressed in the existing literature.

An improved minimal path selection approach with new strategies for pavement crack segmentation

Intelligent pavement detection technology provides a new research idea for identifying pavement cracks. To solve the problem that traditional minimal cost path selection (MPS) is computationally intensive and inefficient in crack segmentation, this study proposes an improved minimal cost path algorithm (IMPS). The contribution of IMPS is reflected in its more efficient path planning than MPS and better choice of source and destination of each path by using new strategies. First, a grayscale pavement image is divided into nonoverlapping cells with a certain size. Next, the darkest points in each cell are selected as candidate points and adjacent candidate points are connected based on IMPS. Then pseudocracks are removed based on postprocessing steps before cracks are well segmented. IMPS is compared with other crack segmentation algorithms to prove its reliability. The efficiency of IMPS is increased by approximately 56% than MPS. The accuracy of IMPS is higher than that of MPS and other methods, with Mean Intersection over Union reaching 73.99%.

Pixel-level pavement crack segmentation with encoder-decoder network

Crack detection is important to pavement condition surveys. The convolutional neural network (CNN) is one of the most powerful tools in computer vision. However, pixel-perfect crack segmentation based on CNNs is still challenging. This paper proposes an encoder-decoder network (EDNet) for crack segmentation to overcome the quantity imbalance between crack and non-crack pixels, which causes many false-negative errors. The decoder of the proposed EDNet is an autoencoder and self-encodes the ground-truth image to corresponding feature maps that are completely abstract, resulting in significantly reduced quantity imbalance between crack and non-crack pixels. Therefore, instead of fitting crack images directly with ground-truth images, EDNet’s encoder fits crack images with corresponding feature maps to overcome the quantity imbalance problem. EDNet achieves overall F1-scores of 97.80% and 97.82% on 3D pavement images and the CrackForest dataset, respectively. Experimental results show that EDNet outperforms other state-of-the-art models.