Unified weakly and semi-supervised crack segmentation framework using limited coarse labels

Abstract

Obtaining extensive, high-quality datasets for crack segmentation with pixel-level labels is expensive and labor-intensive. The Unified Weakly and Semi-supervised Crack Segmentation (UWSCS) framework addresses this challenge by leveraging a limited number of images with coarse labels and a larger set of unlabeled images. Two label correction modules, based on super-pixel segmentation and a shrink module, are incorporated in the model training to improve crack label accuracy and optimize edge refinement. UWSCS employs a dual-encoder fusion network, combining transformers and convolutional neural networks, to enhance crack segmentation in complex backgrounds. An enhanced algorithm using the medial axis transform is proposed for accurately quantifying crack length and width. Extensive experiments were conducted on both synthetic and real crack datasets to validate the superior performance of UWSCS. The results underscore the significant impact of label quality and quantity used in training on model prediction accuracy. Trained on a concrete crack dataset with limited coarse labels, UWSCS achieves an Intersection of Union (IoU) of 77.53%, surpassing the fully supervised model using the same number of coarse labels by 28.64%. It closely approaches the performance of a fully supervised model with the same number of fine labels (IoU of 80.21%). UWSCS outperforms other advanced networks and semi-supervised/weakly supervised algorithms when trained with a limited set of more cost-effective manually labeled coarse labels. Integrated with the crack segmentation network, super-pixel segmentation, and shrink modules during training, UWSCS with limited coarse labels performs similarly to a fully supervised model using fine labels, thereby reducing manual labeling costs by over 90% and enhancing detection efficiency in practical engineering.