Nuts&bolts: YOLO-v5 and image processing based component identification system

Abstract

Assembly is an important process of aerospace industry which involves significant number of fastening elements such as bolts, washers and nuts, etc. Presently, the identification of these elements is done manually by humans. Since human operators are prone to errors, any fault in the assembly line can have a major impact on the overall efficiency and safety of an aerospace vehicle. In this paper, we present a deep learning and image processing-based approach for identification of mechanical fixation/fastening elements used in the aerospace assembly line. We propose YOLO-v5 algorithm to classify the components based on their head and lateral shape. We also propose an image processing method to estimate the spatial dimensions of the assembly line components including thread pitch. Moreover, this study also features the development of an image acquisition platform with two cameras mounted and proper lighting mechanism to capture quality images. Despite the challenges associated with such systems, the proposed deep learning (YOLO-v5) algorithm has shown promising results with a mAP@0.5 of 0.996 for component classification. Whereas, the proposed image processing mechanism achieved an accuracy of 100% for standard size assignments with a maximum error of 0.05 mm for pitch calculation.