SparseShift-GCN: High precision skeleton-based action recognition

Abstract

Skeleton-based action recognition is widely used due to its advantages of lightweight and strong anti-interference. Recently, graph convolutional networks (GCNs) have been applied to action recognition and have made breakthrough progress. The shift convolution operator can effectively replace the spatial convolution and greatly reduce the computational complexity of the algorithm. This article first applies the Conv-Shift-Conv (CSC) module and the Shift-Conv-Shift-Conv (SC2) module to replace the Shift-Conv-Shift (SCS) module in spatial graph convolution of Shift-GCN respectively. This design can reorder the shifted channels more effectively. The experimental results show that the CSC module has the best effect and effectively improves accuracy of model. After that, this article proposes to replace the shift module in the original Shift-GCN with a sparse shift module and named SparseShift-GCN. This structure can reduce the redundancy of features, prevent overfitting and improve the generality of the model. Based on the improvement in the previous step, better results have been achieved. Finally, this paper uses OHEM Loss and Weighted Loss to carefully design the loss function of the model and introduces it into the model proposed in this paper. Experimental results show that OHEM Loss further improves the accuracy of algorithm. After a series of improvements, our proposed model has improved the accuracy of 4 different streams to varying degrees, which improves the overall performance of the network.