Adaptive Graph Convolutional Network With Adversarial Learning for Skeleton-Based Action Prediction

Abstract

The purpose of action prediction is to recognize an action before it is completed to reduce recognition latency. Because action prediction has lower latency than action recognition, it can be applied to a variety of surveillance scenarios and responds faster. However, action prediction is more difficult because it cannot obtain the complete action execution. In this article, we study the action prediction which is based on skeleton data and propose a new network called adaptive graph convolutional network with adversarial learning (AGCN-AL) for it. The AGCN-AL uses adversarial learning to make the features of the partial sequences as similar as possible to the features of the full sequences to learn the potential global information in the partial sequences. Besides, partial sequences with different numbers of frames contain different amounts of information. We introduce temporal-dependent loss functions to prevent the network from paying too much attention to partial sequences whose observation ratios are small, and ignoring partial sequences whose observation ratios are large. Moreover, the AGCN-AL is combined with the local AGCN into a two-stream network to enhance the prediction, proving that the local information and the potential global information in partial sequences are complementary. We evaluate the proposed approach on two data sets and show excellent performance.