Recognition of action dynamics in fencing using multimodal cues

Abstract

Most current approaches to action recognition follow strategies, which permit classification of significantly different actions. However, in some sports disciplines, actions may be distinguished mainly by the dynamics of the motion rather than the trajectory. In this work, we propose a novel approach for recognition of sports actions. The novelty consists in the use of dynamics in the analysis of similar motion patterns. We propose informative motion descriptors based on accelerometric data, skeleton joints features and depth maps, and demonstrate their potential to model the motion dynamics. We show that fusing data from multiple modalities permits better recognition accuracy. We make publicly available a dedicated dataset with fencing footwork samples of ten fencers that consists of depth, skeletal and inertial data of six types of dynamic actions, most of which have similar average trajectories but different dynamics of the motion. We show that on our Fencing Footwork Dataset the proposed method outperforms current state-of-the-art methods for general action recognition.