Dual-referenced assistive network for action quality assessment

Abstract

Action quality assessment (AQA) aims to evaluate the performing quality of a specific action. It is a challenging task as it requires to identify the subtle differences between the videos containing the same action. Most of existing AQA methods directly adopt a pretrained network designed for other tasks to extract video features, which are too coarse to describe fine-grained details of action quality. In this paper, we propose a novel Dual-Referenced Assistive (DuRA) network to polish original coarse-grained features into fine-grained quality-oriented representations. Specifically, we introduce two levels of referenced assistants to highlight the discriminative quality-related contents by comparing a target video and the referenced objects, instead of obtrusively estimating the quality score from an individual video. Firstly, we design a Rating-guided Attention module, which takes advantage of a series of semantic-level referenced assistants to acquire implicit hierarchical semantic knowledge and progressively emphasize quality-focused features embedded in original inherent information. Subsequently, we further design a couple of Consistency Preserving constraints, which introduce a set of individual-level referenced assistants to further eliminate score-unrelated information through more detailed comparisons of differences between actions. The experiments show that our proposed method achieves promising performance on the AQA-7 and MTL-AQA datasets.