A New Multilabel Recognition Framework for Transmission Lines Bolt Defects Based on the Combination of Semantic Knowledge and Structural Knowledge

Abstract

Automatic inspection of transmission lines has received extensive attention in recent years. Bolts are widely used as fasteners in transmission lines, and their defect images have the characteristics of large intra-class differences and small inter-class differences. In view of this, we proposed a multilabel image recognition framework for bolt defects (RFBD). The framework consists of a visual feature and semantic knowledge network (VFSKnet) and a visual feature and position knowledge network (VFPKnet). To learn the relationship between bolt labels, the VFSKnet utilizes the relation between bolt defect labels as professional posterior knowledge to guide the model. To capture the structured fine-grained features, the VFPKnet extracts and utilizes the significant structural features of bolts. Finally, these two sub-networks, VFSKnet and VFPKnet, are combined into a framework after weighting, i.e. RFBD, for the final recognition. By using the proposed RFBD, the label-level accuracy reaches 93.91%, and the image-level accuracy reaches 83.29%. Experimental results well demonstrate the superiority of the proposed model for bolt defect recognition.