A hybrid deep learning pavement crack semantic segmentation

Abstract

Automatic pavement crack segmentation plays a critical role in the field of defect inspection. Although recent segmentation-based CNNs studies showed a promising pavement crack segmentation performance, a challenging task is still remaining due to the inhomogeneity of crack intensity, the complexity of pavement environments, a limited number of labeled training datasets, and the presence of noise in the majority of images in the crack dataset, which makes it difficult to distinguish the cracks from the noise. To overcome these challenges, this study aims to exploit the inherent relationship between classification and segmentation tasks to improve pavement crack segmentation. A hybrid deep learning pavement crack semantic segmentation is proposed based on knowledge transfer among the Class Activation Maps (KTCAM) and the encoder–decoder segmentation network (KTCAM-Net) via the capability of the revised CAM strategy. An adopted trained pavement crack classification network is used to produce high-quality crack localization maps. These localization abilities are fused with the encoder’s image features and fed into the decoder network to provide an accurate pavement crack segmentation. Furthermore, a hybrid loss function is developed to optimize the segmentation network, capture information about thin cracks, and tackle the problem of class imbalance. Afterward, a novel refinement process is applied to purify the segmented crack boundaries. A comprehensive experimental comparison is conducting using four benchmark datasets: DeepCrack, Crack500, CFD, and CrackSC. The proposed KTCAM-Net demonstrates the state-of-the-art segmentation results.