Categorization of human actions with high dynamics in upper extremities based on arm pose modeling

Abstract

This paper proposes a novel method to categorize the human actions with high dynamics in upper extremities. It combines generative and discriminative approaches to infer possible arm pose candidates from images and validate their action categories. The validated action can also facilitate deriving the estimated arm poses. The proposed method exploits the complementary relationship between action categorization and arm pose modeling by adopting arm pose prior of hypothetical action category to enhance modeling possible arm poses, and then applying features captured from temporal and spatial action characteristics of arm pose candidates to improve categorization. From a given visual observation, arm pose states can be estimated on a graphical model via dynamic programming under action category hypothesis, which can be validated by a trained discriminative model based on temporal arm pose words from the estimated arm pose candidates. The proposed method has been evaluated by videos of four action types from the Berkeley multimodal human action dataset with categorization success rate of 91.47 and 95.83 % for single and multiple frames, respectively, and images of three action types from the HumanEva-I dataset with categorization success rate of 96.67 %. Its arm pose modeling performance also has improvement for the actions with high dynamics in upper extremities.