Crack Dataset Research Articles

Automatic Damage Detection and Diagnosis for Hydraulic Structures Using Drones and Artificial Intelligence Techniques

Large-volume hydraulic concrete structures, such as concrete dams, often suffer from damage due to the influence of alternating loads and material aging during the service process. The occurrence and further expansion of cracks will affect the integrity, impermeability, and durability of the dam concrete. Therefore, monitoring the changing status of cracks in hydraulic concrete structures is very important for the health service of hydraulic engineering. This study combines computer vision and artificial intelligence methods to propose an automatic damage detection and diagnosis method for hydraulic structures. Specifically, to improve the crack feature extraction effect, the Xception backbone network, which has fewer parameters than the ResNet backbone network, is adopted. With the aim of addressing the problem of premature loss of image detail information and small target information of tiny cracks in hydraulic concrete structures, an adaptive attention mechanism image semantic segmentation algorithm based on Deeplab V3+ network architecture is proposed. Crack images collected from concrete structures of different types of hydraulic structures were used to develop crack datasets. The experimental results show that the proposed method can realize high-precision crack identification, and the identification results have been obtained in the test set, achieving 90.537% Intersection over Union (IOU), 91.227% Precision, 91.301% Recall, and 91.264% F1_score. In addition, the proposed method has been verified on different types of cracks in actual hydraulic concrete structures, further illustrating the effectiveness of the method.

A Crack Segmentation Model Combining Morphological Network and Multiple Loss Mechanism.

With the wide application of computer vision technology and deep-learning theory in engineering, the image-based detection of cracks in structures such as pipelines, pavements and dams has received more and more attention. Aiming at the problems of high cost, low efficiency and poor detection accuracy in traditional crack detection methods, this paper proposes a crack segmentation network by combining a morphological network and a multiple-loss mechanism. First, for improving the identification of cracks with different resolutions, the U-Net network is used to extract multi-scale features from the crack image. Second, for eliminating the effect of polarized light on the cracks under different illuminations, the extracted crack features are further morphologically processed by a white-top hat transform and a black-bottom hat transform. Finally, a multi-loss mechanism is designed to solve the problem of the inaccurate segmentation of cracks on a single scale. Extensive experiments are carried out on five open crack datasets: Crack500, CrackTree200, CFD, AEL and GAPs384. The experimental results showed that the average ODS, OIS, AIU, sODS and sOIS are 75.7%, 73.9%, 36.4%, 52.4% and 52.2%, respectively. Compared with state-of-the-art methods, the proposed method achieves better crack segmentation performance. Ablation experiments also verified the effectiveness of each module in the algorithm.

Tongue crack recognition using segmentation based deep learning

Tongue cracks refer to fissures with different depth and shapes on the tongue’s surface, which can characterize the pathological characteristics of spleen and stomach. Tongue cracks are of great significance to the objective study of tongue diagnosis. However, tongue cracks are small and complex, existing methods are difficult to extract them effectively. In order to achieve more accurate extraction and identification of tongue crack, this paper proposes to apply a deep learning network based on image segmentation (Segmentation-Based Deep-Learning, SBDL) to extract and identify tongue crack. In addition, we have studied the quantitative description of tongue crack features. Firstly, the pre-processed tongue crack samples were amplified by using adding salt and pepper noise, changing the contrast and horizontal mirroring; secondly, the annotation tool Crack-Tongue was used to label tongue crack; thirdly, the tongue crack extraction model was trained by using SBDL; fourthly, the cracks on the tongue surface were detected and located by the segmentation network, and then the output and features of the segmentation network were put into the decision network for the classification of crack tongue images; finally, the tongue crack segmentation and identification results were quantitatively evaluated. The experimental results showed that the tongue crack extraction and recognition results based on SBDL were better than Mask Region-based Convolutional Neural Network (Mask R-CNN), DeeplabV3+, U-Net, UNet++ and Semantic Segmentation with Adversarial Learning (SegAN). This method effectively solved the inaccurate tongue crack extraction caused by the tongue crack’s color being close to the surrounding tongue coating’s color. This method can achieve better tongue crack extraction and recognition results on a small tongue crack data set and provides a new idea for tongue crack recognition, which is of practical value for tongue diagnosis objectification.

A pavement crack synthesis method based on conditional generative adversarial networks.

A Generative Adversarial Network (GAN) based asphalt pavement crack image generation method was proposed to improve the dataset size of the road images. Five open-source road crack datasets were leveraged to construct an image dataset, which contained two labels - transverse cracks and longitudinal cracks. The constructed dataset was used to facilitate crack detection and classification research by providing a diverse collection of labeled crack images derived from multiple public sources. The network structure of fully connected, convolutional and attention mechanisms based on the Conditional Generative Adversarial Network (CGAN) was used in this project. The purpose of this study was to train a generative model on selected categories of input pavement crack images and generate realistic crack images of those categories. We aim to tune the parameters of the GAN and optimize hyperparameters to improve the realism possibility of generated images. It also explored the generated images with different sizes and evaluated the performance of networks with different architectures. In particular, we analyzed the structural characteristics of conditional GAN. Results demonstrated that the Self-Attention Generative Adversarial Networks (SAGAN) model, which combines self-attention mechanisms with CGAN, can effectively address challenges related to limited crack image data and the inability to selectively generate images from specific categories. By conditioning the generator on category information, the SAGAN model was able to generate high-quality images while focusing on the target categories. Overall, the self-attention and conditional aspects of the SAGAN framework helped improve the generation of realistic pavement crack images.

Dam Crack Image Detection Model on Feature Enhancement and Attention Mechanism

Dam crack detection can effectively avoid safety accidents of dams. To solve the problem that the dam crack image samples are not available and the traditional algorithm detects cracks with low accuracy, we provide a dam crack image detection model based on crack feature enhancement and attention mechanism. Firstly, we expand the dam crack image dataset through a generative adversarial network based on crack feature enhancement (Cracks Enhancements GAN, CE-GAN). It can fully expand the dam crack data samples and improve the quality of the training data. Secondly, we propose a crack image detection model based on the attention mechanism (Attention-based Faster-RCNN, AF-RCNN). The attention mechanism is added in the crack detection module to give different weights to the proposal boxes around the crack target and fuse the candidate boxes with high weights to accurately detect the crack target location. The experimental results show that our algorithm achieves 81.07% mAP on the expanded dam crack dataset, which is 8.39% higher than the original Faster-RCNN algorithm. The detection accuracy is significantly improved compared with other traditional dam crack detection algorithm models.

Concrete crack detection using lightweight attention feature fusion single shot multibox detector

As one of the most important defects of concrete, cracks seriously threaten the service life and safety of concrete structures, and various safety incidents caused by the collapse of concrete structures have occurred. Therefore, it is essential to detect concrete cracks as soon as possible. Existing object detection methods have low detection accuracy for cracks, leading to unsatisfactory detection results. In this paper, we propose a variety of feasible modules that improve the accuracy of single shot multibox detection (SSD), which is the most efficient object detection method in terms of both accuracy and speed. First, to improve the neural network’s ability to learn high-level and low-level feature maps, we propose a feature fusion enhancement module (FFEM). Second, to more effectively capture the information between feature map channels, we propose convolutional network attention (CNA). Third, to improve the anchor box fit to the ground truth box, we reset the distribution of the anchor box. Last, we propose a new type of nonmaximum suppression (NMS) named T-Soft NMS to address numerous issues with current NMS and to significantly enhance the performance of the model. We tested our method on a crack dataset, and numerous tests showed that it outperformed competing methods. In addition, we carried out ablation studies to confirm the validity and efficacy of our method.

Unifying transformer and convolution for dam crack detection

Cracks are a serious disease that threatens the safety of a hydraulic dam. However, detecting dam cracks in time is still a challenging task. It was found that the existing methods fail to detect elongated cracks, leading to discontinuous crack detection results. Moreover, the detection accuracy is severely affected due to the diversity and complexity of cracks and backgrounds. To address these problems, we propose a pixel-level crack detection network by unifying the transformer and CNN models (UTCD-Net). Specifically, the transformer model is designed to extract the global context features for detecting long-range cracks and removing distracting background information, the CNN model is employed to extract local feature information for the detection of thin cracks, and the final segmentation result is refined by the attention fusion module. In comparison with current mainstream methods, the proposed method can capture both local and global features of cracks, which are helpful to detect thin and long cracks. The method obtained promising results on our self-built dam dataset in terms of the Intersection of Union and F1 score (IoU:64.08%, F1:49.43%). Moreover, the experimental results on the other three public crack datasets demonstrate that the proposed method is flexible for adapting to different scenarios. The self-built dam crack dataset provides a challenging benchmark for future research.

U2CrackNet: a deeper architecture with two-level nested U-structure for pavement crack detection

Crack detection is important for evaluating pavement conditions. In this paper, we propose an automatic pavement crack detection method called as U2CrackNet. The architecture of U2CrackNet is a two-level nested U-structure which is an encoding and decoding architecture. Firstly, the crack features are extracted by the encoding layer so that the preliminary effective feature layer is obtained. Subsequently, the encoder and decoder are connected by an atrous spatial pyramid pooling (ASPP) model. The atrous convolution with different expansion rates can be used to capture the multi-scale crack information. Besides, in order to make the network pay more attention to the features of cracks, the crack feature map channels are given different weights through the effective channel attention mechanism after upsampling in the decoding layer. Finally, the crack saliency probability maps generated by each layer of the network are fused into the final crack saliency map through cascade operation. The proposed method has been evaluated on an expanded pavement crack dataset containing 8700 images. The experimental results demonstrate that the proposed U2CrackNet can obtain more clear and continuous cracks. Specifically, the precision, accuracy, F1-score, and Mean Intersection over Union (MIoU) are 89.51%, 98.95%, 81.45%, and 69.19%, respectively.

A Novel Multi-Scale Feature Fusion-Based 3SCNet for Building Crack Detection

Crack detection at an early stage is necessary to save people’s lives and to prevent the collapse of building/bridge structures. Manual crack detection is time-consuming, especially when a building structure is too high. Image processing, machine learning, and deep learning-based methods can be used in such scenarios to build an automatic crack detection system. This study uses a novel deep convolutional neural network, 3SCNet (3ScaleNetwork), for crack detection. The SLIC (Simple Linear Iterative Clustering) segmentation method forms the cluster of similar pixels and the LBP (Local Binary Pattern) finds the texture pattern in the crack image. The SLIC, LBP, and grey images are fed to 3SCNet to form pool of feature vector. This multi-scale feature fusion (3SCNet+LBP+SLIC) method achieved the highest sensitivity, specificity, an accuracy of 99.47%, 99.75%, and 99.69%, respectively, on a public historical building crack dataset. It shows that using SLIC super pixel segmentation and LBP can improve the performance of the CNN (Convolution Neural Network). The achieved performance of the model can be used to develop a real-time crack detection system.

A Fractional Integral and Fractal Dimension-Based Deep Learning Approach for Pavement Crack Detection in Transportation Service Management

With artificial intelligence prevailing in intelligent transportation system, pavement crack detection with deep learning has aroused wide attentions in both academia and transportation sector. Nevertheless, it still remains a challenge to accomplish crack detection due to the complexity in pavement background. Motivated by latest advents in computer vision research, a fractional integral-based filtering method is advocated to remove pavement noise, and a fractal dimension estimation method has also emerged to present shape feature at pixel level, with the multi-scale feature architecture. Therefore, we try to propose a deep learning method, integrating fractional integral with fractal dimension, for crack detection in transportation service management. Firstly, the crack image is taken as input in the bottom-up architecture to extract fractal dimension on multi-scale levels, and a per-level feature unit is built to incorporate maps to make context information flow. Secondly, after fed into a convolutional filter for dimension resizing, all the resized feature maps are next fused at each level to comprise a group network. Finally, extensive experiments are executed on different crack datasets, exhibiting that the proposed method surpasses existing cutting-edge ones in terms of both generalizability and accuracy, with the benefits from not only fractional integral filtering, but also multi-scale fractal dimension features.

The classification and localization of crack using lightweight convolutional neural network with CBAM

Convolutional Neural Networks (CNNs) are currently often used for crack detection. However, the crack datasets collected in real engineering are imbalanced datasets and are affected by interference factors such as different illumination issues and the coexistence of various material crack images. Therefore, the generalization ability of the model itself and the ability to face imbalanced datasets is critical. In addition, a real engineering environment is usually low computational power environment. Therefore, it is undoubtedly more beneficial for the model to have a lightweight feature for practical applications.To address the above challenges in crack detection, MobileNetV3-Large is employed as the backbone combined with CBAM (Convolutional Block Attention Module) to gain MobileNetV3-Large-CBAM in this study. The classification and identification of crack are studied by using the open-source bridge crack dataset. MobileNetV3-Large-CBAM is compared with cutting-edge CNNs, and it verifies that the proposed model combined with the preferred Focal Loss has good performance in dealing with imbalanced datasets and hard samples. To verify the generalization ability of the proposed model, this paper further studies the crack datasets with various material and huge-width cracks under different illumination issues. Finally, the sliding window is adopted to perform crack detection and localization on the three randomly reconstructed crack images.The research results show that, compared with other CNNs, the proposed lightweight MobileNetV3-Large-CBAM combined with the preferred Focal Loss has better comprehensive performance, and the model size is 16.6 MB. I. For imbalanced datasets, the proposed model obtains the best results for crack classification. The Overall Accuracy (OA), F1-score, training speed, and classification speed of MobileNetV3-Large-CBAM are 95.90 %, 95.89 %, 101 images/second and 48 images/second, respectively. The proposed model has a balanced recognition accuracy for different crack categories, and the recognition accuracy for hard samples-irregular crack reaches 94.40 %. II. The proposed model has excellent generalization ability. For two test sets - various material cracks and huge-width cracks under different illumination issues, the OA of MobileNetV3-Large-CBAM reaches 99.66 % and 99.69 %, respectively, the accuracy of crack identification is 99.50 % and 100.00 %, and the accuracy of non-crack identification is 99.90 % and 99.50 %, respectively. III. For crack detection and localization, the model proposed in this paper combined with a sliding window, the accuracy of crack detection for three reconstructed images achieves 100 %, and the average crack localization accuracy achieves 98.40 %.

Super-Resolution Reconstruction Method of Pavement Crack Images Based on an Improved Generative Adversarial Network.

A super-resolution reconstruction approach based on an improved generative adversarial network is presented to overcome the huge disparities in image quality due to variable equipment and illumination conditions in the image-collecting stage of intelligent pavement detection. The nonlinear network of the generator is first improved, and the Residual Dense Block (RDB) is created to serve as Batch Normalization (BN). The Attention Module is then formed by combining the RDB, Gated Recurrent Unit (GRU), and Conv Layer. Finally, a loss function based on the L1 norm is utilized to replace the original loss function. The experimental findings demonstrate that the self-built pavement crack dataset's Peak Signal-to-Noise Ratio (PSNR) and Structural Similarity (SSIM) of the reconstructed images reach 29.21 dB and 0.854, respectively. The results improved compared to the Set5, Set14, and BSD100 datasets. Additionally, by employing Faster-RCNN and a Fully Convolutional Network (FCN), the effects of image reconstruction on detection and segmentation are confirmed. The findings indicate that the segmentation results' F1 is enhanced by 0.012 to 0.737 and the detection results' confidence is increased by 0.031 to 0.9102 when compared to state-of-the-art methods. It has a significant engineering application value and can successfully increase pavement crack-detecting accuracy.

An improved U-Net model for concrete crack detection

Crack detection plays an important role in disease assessment of concrete buildings. However, factors such as complex background, irregular edge, and the real-time and accuracy requirement also make crack detection a challenging task. Aiming at the above challenges, an improved U-Net model for concrete crack detection is proposed, which has strong capability to extract the linear object, improving the performance in crack detection. The model is named Residual Linear Attention U-Net (RLAU-Net). There are three key measures in this paper. First, mirror padding the source image before convolution. Second, the multi-level features are obtained by aggregating the multi-scale features level by level. Third, strip pooling kernels are used to extract global contextual information, reducing information interference from the background. We tested the performance of RLAU-Net on our crack dataset, and the experimental results exhibited that it can improve the quantitative results of mean Intersection Over Union to 81.69%. In addition, F1 score has increased to, 78.21%, the Intersection Over Union of crack increased to 64.47%. We also compared the detect time-consuming of RLAU-Net and that of the original U-Net. Results demonstrate that the proposed model has a short processing time while maintaining a high detection accuracy for crack detection.

OASIS-Net: Morphological Attention Ensemble Learning for Surface Defect Detection

Surface defect detection systems, which have advanced beyond conventional defect detection methods, lower the risk of accidents and increase working efficiency and productivity. Most fault detection techniques demand extra tools, such as ultrasonic sensors or lasers. With the advancements, these techniques can be examined without additional tools. We propose a morphological attention ensemble learning for surface defect detection called OASIS-Net, which can detect defects of three kinds (crack, efflorescence, and spalling) at the bounding box level. Based on the morphological analysis of each defect, OASIS-Net offers specialized loss functions for each defect that can be examined. Specifically, high-frequency image augmentation, connectivity attention, and penalty areas are used to detect cracks. It also compares the colors of the sensing objects and analyzes the image histogram peaks to improve the efflorescence-verification accuracy. Analyzing the ratio of the major and minor axes of the spalling through morphological comparison reveals that the spalling-detection accuracy improved. Defect images are challenging to obtain due to their properties. We labeled some data provided by AI hub and some concrete crack datasets and used them as custom datasets. Finally, an ensemble learning technique based on multi-task classification is suggested to learn and apply the specialized loss of each class to the model. For the custom dataset, the accuracy of the crack detection increased by 5%, the accuracy of the efflorescence detection increased by 4.4%, and the accuracy of the spalling detection increased by 6.6%. The experimental results reveal that the proposed network outperforms the previous state-of-the-art methods.

Efficient and Refined Deep Convolutional Features Network for the Crack Segmentation of Solar Cell Electroluminescence Images

High-quality and fast crack segmentation in solar cell electroluminescence images with heterogeneously textured background disturbance is challenging. We propose an end-to-end Efficient and Refined Deep Convolutional Features Network (ERDCF-Net) for precise and efficient crack segmentation. Firstly, we design a lightweight, efficient deep convolutional features (EDCF) encoder by adopting multilevel deep convolutional features to generate a discriminative feature representation of cracks. Moreover, some complicated background disturbance is restrained by reusing rich crack and background features. Secondly, we present a refined side output (RS) decoder by automatically refining side outputs from the EDCF encoder. The experiments demonstrate that the proposed network excellently performs on the solar cell cracks (MIoU of 92.82%, F - measure of 93.58%, and 89 FPS) and open crack datasets (MIoU of 85.6%, F - measure of 84.1%, and 90 FPS).

Method of convolutional neural network with hybrid attention for crack detection

In recent years, automatic image crack detection has become a critical task for ensuring the safety of various facilities. Many researchers utilized convolutional neural networks (CNNs) for crack detection. However, the existing CNN methods have the limitation of the receptive fields and struggle to establish long dependencies and global background information. Aiming to address this problem, we propose a CNN method with an attention module and recurrent mechanism. In terms of the backbone network, we propose a backbone network named Attention-VGG-19. In our network, we remove the last three fully connected layers of VGG-19 to expect the meaningful side output with different scales and to reduce the number of parameters. The fully connected layers are computationally intensive. The proposed network also provides integrated direct supervision for features of each convolutional stage. Then, to promote important features and suppress unimportant features, an attention module called CrackP attention is proposed. In the recurrent mechanism, the module can be unrolled into R loops to connect the path between one pixel and its neighboring ones. To prove the effectiveness of our proposed method, we evaluate it on four public crack datasets, DeepCrack, crack forest dataset (CFD), Crack500, and CrackTree260, on which it achieves F-score (F1) values of 0.876, 0.652, 0.692, and 0.593, respectively. After a guided filter is added, our method achieves F1 values of 0.878, 0.638, 0.681, and 0.526 on these four datasets.

A deep learning-based fatigue crack growth rate measurement method using mobile phones

This paper proposes a fatigue crack growth rate (FCGR) measure method based on deep learning. Crack data sets of different scales are first collected by a camera, and are then used to train the faster region-based convolutional neural network (Faster R-CNN), respectively. A novel global and local dual-scale Faster R-CNN is further constructed and integrated into the mobile phone to predict the crack length during the entire loading cycle. The fatigue crack growth rate constants are finally obtained by integrating fracture mechanics. The proposed method can effectively measure the propagating small cracks during cyclic loading and extend to non-standard specimens.

Automatic Pavement Crack Identification Based on an Improved C-Mask Region-Based Convolutional Neural Network Model

A pavement crack identification method based on an improved C-mask region-based convolutional neural network (R-CNN) model is proposed to solve problems whereby existing crack recognition algorithms exhibit low accuracy and cannot perform detection and segmentation tasks simultaneously. The crack dataset that was collected in this study included three categories: transverse cracks, longitudinal cracks, and alligator cracks. The model integrates the detection task and the segmentation task into one model, and segments the crack pixels in the generated detection box while achieving target positioning. Firstly, based on the mask R-CNN model, the improved C-mask R-CNN method is designed, which improves the quality of the region proposal box by combining the detectors that are cascaded with different intersections over union thresholds, and achieves accurate crack location under high-threshold detection. Secondly, the ratio of the anchor in the model is adjusted, and a series of optimization parameters and experimental comparisons are carried out for the improved model to realize the segmentation of the crack pixels in the generated detection box during the crack location. The effectiveness of the proposed model is verified, and finally, an evaluation method for cracks is proposed. Furthermore, the calculation of the crack geometric parameters is completed. The experimental results demonstrate that the mean average precision (mAP) of the C-mask R-CNN model detection part reached 95.4%, and the mAP of the segmentation part reached 93.5%. Moreover, the proposed model is convenient for researchers to deploy and implement.

An Efficient Method for Detecting Asphalt Pavement Cracks and Sealed Cracks Based on a Deep Data-Driven Model

Thanks to the development of deep learning, the use of data-driven methods to detect pavement distresses has become an active research field. This research makes four contributions to address the problem of efficiently detecting cracks and sealed cracks in asphalt pavements. First, a dataset of pavement cracks and sealed cracks is created, which consists of 10,400 images obtained by a vehicle equipped with a highway condition monitor, with 202,840 labeled distress instances included in these pavement images. Second, we develop a dense and redundant crack annotation method based on the characteristics of the crack images. Compared with traditional annotation, the method we propose generates more object instances, and the localization is more accurate. Next, to achieve efficient crack detection, a semi-automatic crack annotation method is proposed, which reduces the working time by 80% compared with fully manual annotation. Finally, comparative experiments are conducted on our dataset using 13 currently prevailing object detection algorithms. The results show that dense and redundant annotation is effective; moreover, cracks and sealed cracks can be efficiently and accurately detected using the YOLOv5 series model and YOLOv5s is the most balanced model with an F1-score of 86.79% and an inference time of 14.8ms. The pavement crack and sealed crack dataset created in this study is publicly available.

FFEDN: Feature Fusion Encoder Decoder Network for Crack Detection

Crack detection plays a crucial role in structural health monitoring tasks to ensure the reliability of the transportation infrastructures. However, the automatic detection of cracks remains a challenging task due to the complicated background. Especially, tiny crack detection should be attached importance because of its weak feature and background interference. Therefore, an end-to-end network Feature Fusion Encoder Decoder Network (FFEDN) with two novel modules is proposed to improve the crack detection accuracy. For one thing, the representation capability for tiny cracks is enhanced by introducing the attention mechanism, which redistributes and fuses different features of both the encoder and the decoder. For another, because high-level feature contains less interference, a shape semantic prior module is developed to learn the shape prior map that provides the rough shape and location information of cracks. This map is fed into the lower-level feature and helps it focus on crack areas, thereby suppressing background interference. To demonstrate the effectiveness of the proposed network, several experiments are implemented on three publicly available crack datasets. Compared with state-of-the-art crack detection methods, the novel network shows better performance on all the six evaluation metrics.