Abstract

Skeleton-based action recognition is a typical classification problem which plays a significant role in human-computer interaction and video understanding. Since a human skeleton has natural graphic features, methods based on graph convolutional networks (GCN) are widely applied in skeleton-based action recognition. Previous studies mainly focus on structural links in GCN to generate high-level features of human skeleton. However, low-level features are also important in many applications. For instance, low-level edge gradient and color information are important for image classification. This paper introduces a multi-branches structure to capture different low-level features of human skeleton. We combine both high-level and low-level features to recognize human action. We validate our method in action recognition with two skeleton datasets, NTU-RGB+D and Kinetics. Experiment results indicate that the proposed method achieves considerable improvement over some state-of-the-art methods.

Highlights

  • H UMAN action recognition is a typical classification problem which plays a key role in computer vision

  • The main contributions of our work lie in three aspects: (1) A human action recognition framework with a multibranches structure is proposed to learn the low-level features of skeleton data

  • Since skeleton videos can be seen as a sequence of frames, methods based on RNN [16] [32] [33] are introduced into action recognition

Read more

Summary

INTRODUCTION

H UMAN action recognition is a typical classification problem which plays a key role in computer vision. The main problem of ST-GCN is that the skeleton graph is predefined and the adjacency matrix represents only the physical structure of a human body. They design an adaptive adjacency matrix to capture links between indirectly connected joints dynamically. Illustration of deep learning frameworks for skeleton-based action recognition.From top to bottom: (a) GCN is used to capture connections in one skeleton in one frame and TCN is used to capture connections on the same joint in different frames. The main contributions of our work lie in three aspects: (1) A human action recognition framework with a multibranches structure is proposed to learn the low-level features of skeleton data. References [19] [20] [21] [22] and this work all adapt the layerwise update rule in [30]

SKELETON-BASED ACTION RECOGNITION
SPATIO-TEMPORAL GCN
ADAPTIVE GCN
LOW-LEVEL FEATURES OF ADAPTIVE GRAPH CONVOLUTIONAL NETWORKS
DATASETS
NETWORK ARCHITECTURE AND TRAINING DETAILS
Methods
Findings
CONCLUSION
Full Text
Published version (Free)

Talk to us

Join us for a 30 min session where you can share your feedback and ask us any queries you have

Schedule a call