Transfer learning-based encoder-decoder model with visual explanations for infrastructure crack segmentation: New open database and comprehensive evaluation

Abstract

Contemporary demands necessitate the swift and accurate detection of cracks in critical infrastructures, including tunnels and pavements. This study proposed a transfer learning-based encoder-decoder method with visual explanations for infrastructure crack segmentation. Firstly, a vast dataset containing 7089 images was developed, comprising diverse conditions—simple and complex crack patterns as well as clean and rough backgrounds. Secondly, leveraging transfer learning, an encoder-decoder model with visual explanations was formulated, utilizing varied pre-trained convolutional neural network (CNN) as the encoder. Visual explanations were achieved through gradient-weighted class activation mapping (Grad-CAM) to interpret the CNN segmentation model. Thirdly, accuracy, complexity (computation and model), and memory usage assessed CNN feasibility in practical engineering. Model performance was gauged via prediction and visual explanation. The investigation encompassed hyperparameters, data augmentation, deep learning from scratch vs. transfer learning, segmentation model architectures, segmentation model encoders, and encoder pre-training strategies. Results underscored transfer learning's potency in enhancing CNN accuracy for crack segmentation, surpassing deep learning from scratch. Notably, encoder classification accuracy bore no significant correlation with CNN segmentation accuracy. Among all tested models, UNet-EfficientNet_B7 excelled in crack segmentation, harmonizing accuracy, complexity, memory usage, prediction, and visual explanation.