Advancing Temporal Action Localization with a Boundary Awareness Network

Abstract

Temporal action localization (TAL) is crucial in video analysis, yet presents notable challenges. This process focuses on the precise identification and categorization of action instances within lengthy, raw videos. A key difficulty in TAL lies in determining the exact start and end points of actions, owing to the often unclear boundaries of these actions in real-world footage. Existing methods tend to take insufficient account of changes in action boundary features. To tackle these issues, we propose a boundary awareness network (BAN) for TAL. Specifically, the BAN mainly consists of a feature encoding network, coarse pyramidal detection to obtain preliminary proposals and action categories, and fine-grained detection with a Gaussian boundary module (GBM) to get more valuable boundary information. The GBM contains a novel Gaussian boundary pooling, which serves to aggregate the relevant features of the action boundaries and to capture discriminative boundary and actionness features. Furthermore, we introduce a novel approach named Boundary Differentiated Learning (BDL) to ensure our model’s capability in accurately identifying action boundaries across diverse proposals. Comprehensive experiments on both the THUMOS14 and ActivityNet v1.3 datasets, where our BAN model achieved an increase in mean Average Precision (mAP) by 1.6% and 0.2%, respectively, over existing state-of-the-art methods, illustrate that our approach not only improves upon the current state of the art but also achieves outstanding performance.