A Domain-Adaptive Segmentation Method Based on Segment Anything Model for Mechanical Assembly

Abstract

Semantic segmentation of assembly images can effectively identify and prevent non-standard behaviors such as wrong assembly and missing assembly in the assembly process. However, supervised methods require massive training samples and are time-consuming to label. Unsupervised domain adaptation methods can address this issue without labeling the target data. Recently, the large-scale Segment Anything Model (SAM) has achieved significant success in semantic segmentation owing to its powerful feature extraction and instant interaction capabilities. Based on this, this paper proposed a new framework called U-SAM for unsupervised domain-adaptive semantic segmentation. Compared to other frameworks, U-SAM can obtain more segmented features while maintaining shallower features. Meanwhile, this study proposed an iterative loop strategy for the prompt module in U-SAM. This strategy gradually improves the accuracy of target prompt labels and solves the problem of no prompt labels in the target data. Additionally, a novel method of using oblique boxes instead of regular boxes was developed to achieve higher box prompt quality. Finally, a line discriminator module and a trainable line-guided filtering module were designed to improve the quality of edge segmentation. The experimental results indicate that the U-SAM network framework achieves a Dice accuracy of 84.4% for semantic segmentation on the mechanical assembly image dataset. This confirmed that our proposed method can be applied to the assembly monitoring process of target data.