Task Of Action Detection Research Articles

Temporal Attention-Pyramid Pooling for Temporal Action Detection

Temporal action detection is a challenging task in video understanding, which is usually divided into two stages: proposal generation and classification. Learning proposal features is a crucial step for both stages. However, most methods ignore temporal information of proposals and consider background and action frames in proposals equally, leading to poor proposal features. In this paper, we propose a novel Temporal Attention-Pyramid Pooling (TAPP) method to learn proposal features of arbitrary length action proposals. The TAPP method exploits the attention mechanism to focus on the discriminative part of proposals, suppressing background influence on proposal features. It constructs a temporal pyramid structure to convert arbitrary length proposal feature sequences to multiple fixed-length sequences while retaining the temporal information. In the TAPP method, we design a multi-scale temporal function and apply it to the temporal pyramid to generate final proposal features. Based on the TAPP method, we construct a temporal action proposal generation model and an action proposal classification model, and then we perform extensive experiments on two mainstream temporal action detection datasets for the temporal action proposal and temporal action detection tasks to verify our models. On the THUMOS'14 dataset, our models based on the TAPP significantly outperform the previous state-of-the-art methods for both tasks.

Modeling long-term video semantic distribution for temporal action proposal generation

Video temporal segmentation plays a vital role in video analysis since many higher-level computer vision tasks rely on it. Some recent efforts have been dedicated to generating temporal action proposals for long and untrimmed videos, which requires methods to generate accurate boundaries for video semantics. In this paper, we propose a novel and efficient Temporal Distribution Network (TDN), to model the long-term distribution of video semantic units (video dictionary). Firstly, we encode the semantics and context relations of video segments with a boundary-specified video embedding method. Then based on temporal convolutional layers, we design a Temporal Distribution Network (TDN) enumerating all the possible temporal locations in one pass and generating proposals that have high action confidence scores by capturing the long-term distributions of video semantics. We validate our method on temporal action proposal generation tasks and action detection tasks. Experimental results on two benchmark datasets, THUMOS14 and ActivityNet-1.3, show that the proposed method can significantly outperform the state-of-the-art approaches. Our model could obtain high-quality action proposals with a much faster speed.

Skeleton‐based attention‐aware spatial–temporal model for action detection and recognition

Action detection and recognition are popular subjects of research in the field of computer vision. The task of action detection can be regarded as the sum of action location and recognition. Action features described by using information concerning the human skeleton have the advantages of robustness against external factors and requiring a small amount of calculation. This study proposes a skeleton‐based action analysis model based on a recurrent neural network framework. The model learns action features by modelling static and dynamic features of skeleton joints and the importance of different video frames by introducing an attention module. For action location, conditional random field loss function is introduced to establish the context dependency of output labels. In the aspect of action recognition, the hierarchical training mechanism with triple loss models action features at coarse‐grained and fine‐grained levels. The authors’ proposed method delivers state‐of‐the‐art results on action location and recognition tasks.

Classification-oriented structure learning in Bayesian networks for multimodal event detection in videos

We investigate the use of structure learning in Bayesian networks for a complex multimodal task of action detection in soccer videos. We illustrate that classical score-oriented structure learning algorithms, such as the K2 one whose usefulness has been demonstrated on simple tasks, fail in providing a good network structure for classification tasks where many correlated observed variables are necessary to make a decision. We then compare several structure learning objective functions, which aim at finding out the structure that yields the best classification results, extending existing solutions in the literature. Experimental results on a comprehensive data set of 7 videos show that a discriminative objective function based on conditional likelihood yields the best results, while augmented approaches offer a good compromise between learning speed and classification accuracy.