Bi-GRU-Attention Enhanced Unsupervised Network for Skeleton-Based Action Recognition

Abstract

AbstractAction recognition is widely used in human-computer interaction, intelligent monitoring and other applications. Recently, more and more researcher pay attention to it. While, skeleton-based action recognition is more and more popular because of its low cost and robustness. Current good performance methods based on skeleton is supervised mostly, which needs large labeled datasets for train. But for some special task, such as the pedestrian action recognition of unmanned driving in real traffic scene, it is difficult to gain the labeled action data because the large cost of time, money and energy. Therefore, we proposed a Bi-GRU-Attention Enhanced Unsupervised Network (BGAEUN) for action recognition based on skeleton sequence. BGAEUN adopts an encoder-decoder network, and adds an attention mechanism in the encode, learns the weights of the hidden state, and obtains the weights of different skeleton nodes, so that it can better characterize actions and provide better skeleton features. For the decoder, using fixed weight and fixed state strategies to weaken the decoder, calculate the loss of the output and input of the decoder, minimize the loss, make it similar to the input, and minimize the reconstruction loss in the training process. BGAEUN exploits the last layer feature of encoder to classify by k-nearest neighbors algorithm. Experiments on large-scale action dataset NTU-RGB + D show BGAEUN achieves a higher recognition accuracy than current most unsupervised skeleton action recognition methods.KeywordsUnsupervised learningAttention mechanismAction recognition