A Dynamic Gesture Recognition Algorithm based on Feature Fusion from RGB-D Sensor

Abstract

The motion process of dynamic gestures is often very complicated, and extracting appropriate features from the sequence of gestures to characterize the gesture is a daunting task. The current dynamic gesture recognition algorithms generally have problems such as complicated computational process and poor robustness to the environment, which cannot meet the requirements of real-time and complex scenarios. How to improve the simplicity and robustness of the algorithm is a very challenging research topic. Since the common problem affecting system performance is descriptors, it is possible to improve the robustness of descriptors to complex environments and simplify feature extraction algorithms to meet the requirements of interactive systems. This paper proposes a new video descriptor (HOG2-9ULBP2), and utilizes the descriptor to complete dynamic gesture recognition. First, the RGB image sequence and the depth image sequence of the gesture are simultaneously captured by Kinect; then, the HOG2 feature of the gesture is extracted from the depth image sequence and the 9ULBP2 feature of the gesture is extracted from the RGB image sequence; next, the two features are fused to represent the motion feature of the dynamic gesture; finally, the fusion feature is fed into the SVM for gesture recognition. The proposed dynamic gesture recognition method is tested in the SKIG dataset. The experimental results show that compared with other gesture recognition methods, the method can achieve a recognition rate of up to 98.51%.