Attentive pooling for group activity recognition

Abstract

In group activity recognition, the hierarchical framework is widely used to represent the relationships between individuals and their corresponding groups and has achieved promising performance. However, existing methods simply employ the max/average pooling in this framework, overlooking the distinct contributions of different individuals to the group activity recognition. In this paper, we propose a new contextual pooling scheme, named attentive pooling, which enables weighted information transition from individual actions to group activity. Using the attention mechanism, attentive pooling is intrinsically interpretable and can embed the member context in the existing hierarchical model. To verify the effectiveness of the proposed scheme, two specific attentive pooling methods, i.e., global attentive pooling (GAP) and hierarchical attentive pooling (HAP), are designed. GAP rewards individuals significant to the group activity, while HAP further considers the hierarchical division by introducing the subgroup structure. Experimental results on the benchmark dataset demonstrate that the proposed scheme is significantly superior over the baseline and comparable to state-of-the-art methods.