Abstract
The traditional human action recognition (HAR) method is based on RGB video. Recently, with the introduction of Microsoft Kinect and other consumer class depth cameras, HAR based on RGB-D (RGB-Depth) has drawn increasing attention from scholars and industry. Compared with the traditional method, the HAR based on RGB-D has high accuracy and strong robustness. In this paper, using a selective ensemble support vector machine to fuse multimodal features for human action recognition is proposed. The algorithm combines the improved HOG feature-based RGB modal data, the depth motion map-based local binary pattern features (DMM-LBP), and the hybrid joint features (HJF)-based joints modal data. Concomitantly, a frame-based selective ensemble support vector machine classification model (SESVM) is proposed, which effectively integrates the selective ensemble strategy with the selection of SVM base classifiers, thus increasing the differences between the base classifiers. The experimental results have demonstrated that the proposed method is simple, fast, and efficient on public datasets in comparison with other action recognition algorithms.
Highlights
Video has become the primary carrier of information owing to the rapid popularization and development of video acquisition equipment and broadband networks
human action recognition (HAR) based on computer vision technology has been extensively used in several fields of human life, such as smart video surveillance [1, 2], human-machine interaction [3], robotics [3], video analytics [4], and human activity recognition [5,6,7,8,9]
Human action recognition based on RGB information encounters multiple challenges as follows: (1) Complex background, occlusion, shadow, scale change, and different lighting conditions will induce tremendous difficulties for recognition, which is the difficulty of action recognition based on RGB. (2) e same action will generate different views from different perspectives. (3) e same action performed by different people will be significantly varied, and two different types of action may have considerable similarity. ese inherent defects of RGB visual information would limit the performance of human action recognition based on RGB information
Summary
Video has become the primary carrier of information owing to the rapid popularization and development of video acquisition equipment and broadband networks. (1) Acquire synchronized RGB, depth, and joint images from the Kinect sensor (2) Convert the input RGB image to grayscale, and extract the improved histogram of the oriented gradient features (3) Compute the depth motion map-based local binary pattern (DMM-LBP) from the depth image, and extract joint-based hybrid joint features (HJFs) from the acquired 3D skeleton image (4) Train the selective ensemble-based support vector machine (SESVM) using the sample sets with combined features (5) Implement the same extraction process to the predicting images during action recognition, enter them into SESVM for recognition, and work out recognition result e major contributions of this paper are summarized as follows:. (2) e improved RGB-based histogram of oriented gradient (RGB-HOG) features is adopted in this paper, which is invariant to geometric and optical deformations of the images
Sign-in/Register to view highlights & summary Talk to us
Join us for a 30 min session where you can share your feedback and ask us any queries you have
Schedule a callDisclaimer: All third-party content on this website/platform is and will remain the property of their respective owners and is provided on "as is" basis without any warranties, express or implied. Use of third-party content does not indicate any affiliation, sponsorship with or endorsement by them. Any references to third-party content is to identify the corresponding services and shall be considered fair use under The CopyrightLaw.
