Multi-Stream Adaptive Graph Convolutional Network Using Inter- and Intra-Body Graphs for Two-Person Interaction Recognition

Abstract

Two-person interaction recognition has become an area of growing interest in human action recognition. The graph convolutional network (GCN) using human skeleton data has been shown to be highly effective for action recognition. Most GCN-based methods focus on recognizing an individual person’s actions on the basis of an intra-body graph. However, many of these methods do not represent the relation between two bodies, making it difficult to accurately recognize human interaction. In this work, we propose multi-stream adaptive GCN using inter- and intra-body graphs (MAGCN-IIG) as a new method of human interaction recognition. To ensure highly accurate human interaction recognition, our method cooperatively utilizes two types of graphs: an inter-body graph and an intra-body graph. The inter-body graph, which is newly introduced in this paper, connects the inter-body joints between two people as well as intra-body connections. The adaptive GCN using the inter-body graph captures the relation of joints between two people, even different types of joints located far away from each other. Further, by implementing a multi-stream architecture, our method simultaneously captures both inter-body and intra-body relations in each of two units that represent the position and motion of people. Experiments on interaction recognition using two large-scale human action datasets, NTU RGB+D and NTU RGB+D 120, showed that our method recognized human interactions more accurately than state-of-the-art methods.