Crack detection of masonry structure based on thermal and visible image fusion and semantic segmentation

Abstract

The integration of visible and thermal images has demonstrated the potential ability to enhance crack segmentation accuracy. However, due to the intricate texture of masonry structures and the challenges posed in precisely aligning these cross-modality images, it is necessary to explore pixel-level alignment and develop a comprehensive dataset to enable deep-learning based methods. Therefore, a dataset, Crack900, including five image types, is developed together with a proposed two-step registration to achieve highly accurate pixel-level alignment. In addition, both Train from Scratch and Transfer Learning (TL) strategies are applied on eleven models to investigate the impact of different fused image types. Our findings reveal that the concatenation strategy markedly improves segmentation accuracy, and the performance of TL depends on the compatibility of channel numbers and domain difference between pre-trained and target models. These findings pave the way for further development of cross-modality in masonry crack segmentation methodologies for structural health monitorin.