Model for deep learning-based skill transfer in an assembly process

Abstract

As the variety of products and manufacturing processes increases, the expansion of flexible training approaches is crucial to support the development of human skills. This study presents a model for skill transfer support that extracts experts’ relevant skills as actions and objects relevant to the action into a computational model for transferring skills. This model engages two modes of deep learning as the groundwork, namely, convolutional neural network (CNN) for action recognition and faster region-based convolutional neural network (R-CNN) for object detection. To evaluate the performance of the proposed model, a case study of the final assembly of a GPU card is conducted. The accuracy of CNN and faster R-CNN are 95.4% and 96.8%, respectively. The goal of this model is to guide junior operators during the assembly by providing step-by-step instructions in performing complex tasks. The present study facilitates flexible training in terms of adapting new skills from skilled operators to naïve operators by deep learning.