Abstract
Human action recognition remains an important yet challenging task. This work proposes a novel action recognition system. It uses a novel multi-view region-adaptive multi-resolution-in-time depth motion map (MV-RAMDMM) formulation combined with appearance information. Multi-stream 3D convolutional neural networks (CNNs) are trained on the different views and time resolutions of the region-adaptive depth motion maps. Multiple views are synthesised to enhance the view invariance. The region-adaptive weights, based on localised motion, accentuate and differentiate parts of actions possessing faster motion. Dedicated 3D CNN streams for multi-time resolution appearance information are also included. These help to identify and differentiate between small object interactions. A pre-trained 3D-CNN is used here with fine-tuning for each stream along with multi-class support vector machines. Average score fusion is used on the output. The developed approach is capable of recognising both human action and human–object interaction. Three public-domain data-sets, namely MSR 3D Action, Northwestern UCLA multi-view actions and MSR 3D daily activity, are used to evaluate the proposed solution. The experimental results demonstrate the robustness of this approach compared with state-of-the-art algorithms.
Highlights
Action recognition is a key step in many amazing applications areas
We propose an adaptive Multi-resolution depth motion map calculated across multiple views with important action information learned through the 3D convolutional neural networks (CNNs) model to provide extra motion-based features that emphasise the significance of moved parts of an action
The region-adaptive depth motion maps (DMMs) (RADMM) templates are calculated across the three temporal resolutions to form the multi-resolution DMM template, referred to as region-adaptive MDMM (RAMDMM)
Summary
Action recognition is a key step in many amazing applications areas. Potential areas of interest are wide. Deep learning-based features extracted using, for example, CNNs have shown great performance over many traditional handcrafted features due to, in simple terms, their capability to learn the important aspects of actions from the huge amount of variation that can potentially occur in images and video sequences. This property has enabled deep learning-based techniques to have improved invariance to, for example, pose, lighting and surrounding clutter [13]. As a part of the success of the deep learning-based methods, many variations in the architectures and approaches have been proposed
Talk to us
Join us for a 30 min session where you can share your feedback and ask us any queries you have
Disclaimer: All third-party content on this website/platform is and will remain the property of their respective owners and is provided on "as is" basis without any warranties, express or implied. Use of third-party content does not indicate any affiliation, sponsorship with or endorsement by them. Any references to third-party content is to identify the corresponding services and shall be considered fair use under The CopyrightLaw.