Human action recognition from RGB-D data using complete local binary pattern

Abstract

Human action recognition is an active research domain in Computer Vision and Pattern Recognition due to the challenges such as inter and intra class variation, background clutter, partial occlusion and changes in scale, viewpoint, lighting, appearance etc. Human action recognition aims at determining the activities of a person or a group of persons, as well as on knowledge about the context within which the observed activities take place. As RGB cameras responds easily to illumination changes and surrounding clutters, the worthwhile RGB Depth (RGB-D) camera sensors (e.g. Kinect) is used to improve the action recognition. This paper aims at classifying Human Actions by integrating salient motion features from both RGB and Depth Camera. The methodology includes Salient Information Map generation from both RGB and depth action sequences signposting the motion significant region of the corresponding action sequence. From the Salient Information Map, Sign, Magnitude and Center descriptors representing Complete Local Binary Pattern was extracted. Then the fusion of features from depth and RGB is carried out by Canonical Correlation Analysis accompanied by dimensionality reduction. Multiclass SVM classifier is used for classifying the features in to various action categories. The experimental analysis of the proposed algorithm was carried with MSR Daily Activity 3D Dataset and UTD-MHAD Action Dataset and the recognition rate of 98.75% and 84.12% was obtained.