A Smart Operator Assistance System Using Deep Learning for Angle Measurement

Abstract

Manual workstations play a critical role in flexible assembly lines by enabling human responses to reconfiguration that is faster than machine responses. As a result, human is more adaptive and sometimes unreplaceable by machines in complex assembly. However, with the increasing complexity of tasks, product quality has become highly susceptible to human error due to increments in operators’ cognitive load. One of the errors that affect assembly quality is the operator’s use of a handheld tool with an unfavorable working angle when handling the workpiece. In this case, an assistive mechanism to remind workers about the wrong working angle is necessary to support the process. To this end, this study proposes an angle monitoring system to inspect the working angle of handheld tools and provide feedback in real time with minimal interruption to the assembly process. The proposed system consists of an angle measurement model and an action recognition model, which are both built using deep-learning-based object detection algorithm. Besides, the system was designed with flexibility that it is applicable to different tools and assembly tasks. A case study on fastening a high-end graphical processing unit card is investigated to evaluate their performance. Results show 95.83% and 99.83% accuracies of the models. In practice, the proposed study is expected to facilitate assembly quality by preventing the failure of angle-related operations in a timely and reliable manner.