Assembly Monitoring Using Semantic Segmentation Network Based on Multiscale Feature Maps and Trainable Guided Filter

Abstract

Timely detection of errors in assembly process is important for ensuring assembly quality and efficiency. Image segmentation technology can be used to monitor mechanical assembly for missing assembly, wrong assembly, assembly sequence, and assembly pose. However, unclear segmentation boundaries and low segmentation accuracy of small parts is the main limit of assembly depth image segmentation. Therefore, this study proposes a skip Fully Convolutional Network (FCN) with Multi-scale Feature maps (MF) and a Trainable Guided Filter (TGF) (skip FCN with MF&TGF for short) for depth image segmentation of mechanical assembly. By adding skip connections in the second maximum pooling layer of FCN, additional low-level features can be obtained, which compensates the problem of insufficient detailed information in feature map prediction. The problem of image segmentation edge blur is mitigated by the integration of a trainable guided filter. Moreover, this study establishes a mechanical assembly depth image dataset. The experimental results show that skip FCN with MF&TGF enhances the segmentation precision of mechanical assembly and significantly improves the segmentation edges of depth images. Furthermore, it shows significant improvement in segmentation performance when segmenting small parts. Finally, compared with other semantic segmentation networks, skip FCN with MF&TGF achieves the best segmentation performance on the mechanical assembly depth image dataset.