Judging the Normativity of PAF Based on TFN and NAN

Abstract

The normativity of workers’ actions during producing has a great impact on the quality of the products and the safety of the operation process. Previous studies mainly focused on the normativity of each single producing action instead of considering the normativity of continuous producing actions, which is defined as producing action flow (PAF) in this paper, during operation process. For this issue, a normativity judging method based on two-LSTM fusion network (TFN) and normativity-aware attention network (NAN) is proposed. First, TFN is designed to detect and recognize the producing actions based on skeleton sequences of a worker during complete operation process, and PAF data in sequential form are obtained. Then, NAN is built to allocate different levels of attention to each producing action within the sequence of PAF, and by this means, an efficient normativity judging is conducted. The combustor surface cleaning (CSC) process of rocket engine is taken as the experimental case, and the CSC-Action2D dataset is established for evaluation. Experiment results show the high performance of TFN and NAN, demonstrating the effectiveness of the proposed method for PAF normativity judging.