Complicated human action understanding by massive-scale graph discovering technique

Abstract

In order to solve the problem that the first frame of human motion prediction is discontinuous, we notice that the prediction time is short due to the influence of uncertain factors such as motion speed and motion amplitude. In this work, an end-to-end model based on bidirectional gating loop unit (GRU) and attention mechanism, biagru-seq2seq, is proposed. The encoder of our deep model adopts bidirectional Gru structure to input data from both positive and negative directions simultaneously. Meanwhile, the decoder part adopts unidirectional GRU structure and adds attention mechanism to encode the output of the encoder into a vector sequence containing multiple subsets. Also, a large-scale graph discovery framework is used to identify the various human action components. Subsequently, the input and output data of the decoder are fed into the residual at the same time. In the designed tensorflow framework, we use human3.6m, the largest open data set of motion capture data at present, to fulfill human motion prediction applications. Comprehensive experimental results have shown that the proposed model can not only substantially reduce the short-term motion prediction error, but also accurately predict multi frame human action recognition.