A transformer and self-cascade operation-based architecture for segmenting high-resolution bridge cracks

Abstract

High-resolution (HR) crack image segmentation is crucial for accurate bridge safety diagnosis. However, accurately segmenting cracks with their slender topology and random distribution in HR images poses significant challenges. Additionally, the restricted memory capacity of graphics cards presents a significant limitation for HR image segmentation. To tackle this demanding task, a approach called Cascade CATransUNet, consisting of a coordinate attention-enhanced transformer architecture with a self-cascaded design, was proposed in this study. Firstly, CATransUNet, a transformer-based multi-scale feature extraction architecture with the embedment of the coordinate attention mechanism, is customized to enhance the extraction of the crack's main contour both horizontally and vertically. Then, a self-cascade refinement operation is introduced on the basis of CATransUNet to reconstruct the details of the extracted crack features from the global and local levels successively. Furthermore, an optimized boundary loss based on the joint cascade loss function is introduced to improve segmentation quality in boundary areas. The necessity and effectiveness of all the proposed improvements were demonstrated through ablation studies conducted on both open-sourced crack dataset and HR crack images collected on-site. Moreover, the advancement of the proposed method was confirmed in a parallel comparative experiment. The self-cascaded transformer architecture achieved impressive performance with mIoU, mBIoU, and DICE scores exceeding 89.83%, 85.78%, and 96.97% respectively, on HR (4 K) images. Finally, the proposed method was tested through an unmanned aerial vehicle (UAV)-based bridge crack inspection task. The utilization of the Cascade CATransUNet enabled the UAV to accurately detect cracks even from considerable distances. This improvement enhances safety and detection efficiency in UAV inspections, ensuring flexibility in selecting flight paths, thus providing technical support for the promotion and application of UAV-based bridge crack detection technology.