A spatiotemporal motion variation features extraction approach for human tracking and pose-based action recognition

Abstract

Video and image analysis technologies have made human action recognition as an interesting field and used in many practical application systems such as visual surveillance systems, healthcare monitoring systems, 3D games and smart home. In this paper, we address the challenges of automatic tracking, detection and recognition of three-dimensional human pose-based actions from sequences of depth maps. Specifically, we have designed our proposed idea into a probabilistic framework: 1) dividing each action into meaningful ordered temporal segments, 2) using pixel-neighboring intensity difference approach to identify human shape region from the confused scenes, 3) introducing a robust spatiotemporal features to extract 3D joints information via local distance features and depth human shape information to get motion variation features and 4) finally these sets of features are merged together and their feature vectors are further reduced and discriminated to get optimal vectors for real-world applications. Next, the augmented features are enhanced and symbolized by Linde-Buzo-Gray clustering algorithm for better action recognition. With these symbols, each action hidden Markov model (HMM) is trained and tested for action recognition. The experimental results demonstrated our proposed approach on three challenging depth video datasets: IM-DepthActions, MSRAction3D and MSRDailyActivity3D, producing significantly improved results over the state-of-the-art methods: especially for those actions that are not easily discernible by the existing methods.