3D-unified spatial-temporal graph for group activity recognition

Abstract

Early group activity recognition is typically conducted in a 2D scenario. This paper proposes a group activity recognition method in a 3D space. In practical applications, differences in camera angle and mutual shielding during the peak flow of people will affect the extraction of individual characteristics and the calculation of individual relationships. Therefore, a method using 3D relationships is proposed. We first detect the 3D backbone of pedestrians and calculate the movement relationship of individuals in the group based on it. Second, we calculate the 3D-unified spatial–temporal graphs (3D-USTG) of the group and the individuals in the group using global-G3D (G-G3D). The experimental results demonstrate that the proposed method can effectively solve the problems caused by occlusion and shooting angle compared with a 2D method, which indicates that the proposed method can achieve better results when applied to intersections of different specifications. Additionally, the proposed method achieves strong performance metrics with two publicly available datasets for group activity recognition.