HFA-GTNet: Hierarchical Fusion Adaptive Graph Transformer network for dance action recognition

Abstract

Dance action recognition is a hot research topic in computer vision. However, current skeleton-based action recognition methods face difficulties in capturing the adequate spatial structure and temporal variations of dance actions, resulting in lower recognition accuracy. In this paper, we propose a Hierarchical Fusion Adaptive Graph Transformer network (HFA-GTNet) for dance action recognition. A Hierarchical Spatial Attention (HSAtt) module is designed to extract different levels of spatial feature information from joint to parts to group, it can effectively learn high-order dependency relationships from local joints to global poses in dance actions. Secondly, to extract the joint variations in dance actions at different speeds, we have designed a Temporal Fusion Attention (TFAtt) module. This module learns the short-term and long-term temporal dependencies among joints across frames. Additionally, to capture the variations in motion patterns and dance styles among different dancers, we introduce an Adaptive Component (AdaptC). Finally, we evaluate our model on two self-built dance datasets, MSDanceAction and InDanceAction, and demonstrate its superior performance compared to other state-of-the-art methods in dance action recognition.