Temporal-Variation Skeleton Point Correction Algorithm for Improved Accuracy of Human Action Recognition

Abstract

The technique of human action recognition can be applied in a number of fields, such as medical rehabilitation, posture in the domain of sports, and emotion perception in counselling. Deep learning action recognition models that consider the continuous changes in the human skeleton can efficiently identify action states, such as rehabilitation processes, posture, and changes in emotion. However, the keypoints of the human skeleton are susceptible to occlusion due to body movement, resulting in a lower confidence level in their positions, which in turn affects the training and execution of human action recognition models. This paper proposes a keypoint correction algorithm based on continuous-time images of temporal variation that can predict and correct the skeleton keypoints with low confidence, based on the previous and subsequent multiple images. The proposed temporal-variation skeleton keypoint correction algorithm can provide accurate feature association training for the locations of the skeleton keypoints. The proposed method is shown to improve the accuracy of human action recognition by at least 30% over three alternative models in the literature: STV-GCN, GCN-NAS, and 2S-AGCN. The proposed algorithm is also improved by at least 30% compared with the above-mentioned methods in the literature, based on an action recognition model with a Gaussian filter.