Abstract

The traditional human action recognition (HAR) method is based on RGB video. Recently, with the introduction of Microsoft Kinect and other consumer class depth cameras, HAR based on RGB-D (RGB-Depth) has drawn increasing attention from scholars and industry. Compared with the traditional method, the HAR based on RGB-D has high accuracy and strong robustness. In this paper, using a selective ensemble support vector machine to fuse multimodal features for human action recognition is proposed. The algorithm combines the improved HOG feature-based RGB modal data, the depth motion map-based local binary pattern features (DMM-LBP), and the hybrid joint features (HJF)-based joints modal data. Concomitantly, a frame-based selective ensemble support vector machine classification model (SESVM) is proposed, which effectively integrates the selective ensemble strategy with the selection of SVM base classifiers, thus increasing the differences between the base classifiers. The experimental results have demonstrated that the proposed method is simple, fast, and efficient on public datasets in comparison with other action recognition algorithms.

Highlights

  • Video has become the primary carrier of information owing to the rapid popularization and development of video acquisition equipment and broadband networks

  • human action recognition (HAR) based on computer vision technology has been extensively used in several fields of human life, such as smart video surveillance [1, 2], human-machine interaction [3], robotics [3], video analytics [4], and human activity recognition [5,6,7,8,9]

  • Human action recognition based on RGB information encounters multiple challenges as follows: (1) Complex background, occlusion, shadow, scale change, and different lighting conditions will induce tremendous difficulties for recognition, which is the difficulty of action recognition based on RGB. (2) e same action will generate different views from different perspectives. (3) e same action performed by different people will be significantly varied, and two different types of action may have considerable similarity. ese inherent defects of RGB visual information would limit the performance of human action recognition based on RGB information

Read more

Summary

Introduction

Video has become the primary carrier of information owing to the rapid popularization and development of video acquisition equipment and broadband networks. (1) Acquire synchronized RGB, depth, and joint images from the Kinect sensor (2) Convert the input RGB image to grayscale, and extract the improved histogram of the oriented gradient features (3) Compute the depth motion map-based local binary pattern (DMM-LBP) from the depth image, and extract joint-based hybrid joint features (HJFs) from the acquired 3D skeleton image (4) Train the selective ensemble-based support vector machine (SESVM) using the sample sets with combined features (5) Implement the same extraction process to the predicting images during action recognition, enter them into SESVM for recognition, and work out recognition result e major contributions of this paper are summarized as follows:. (2) e improved RGB-based histogram of oriented gradient (RGB-HOG) features is adopted in this paper, which is invariant to geometric and optical deformations of the images

Results
Recognition Method
Experimental Results
16 Dataset G3D CAD60
Recognition methods
Full Text
Published version (Free)

Talk to us

Join us for a 30 min session where you can share your feedback and ask us any queries you have

Schedule a call