Gesture recognition based on HMM-FNN model using a Kinect

Abstract

Addressing the problem of complex dynamic gesture recognition, this paper obtains the body depth image through the body feeling sensor device—Kinect; the threshold segmentation method is used to segment the gestures depth image, on the basis of the common distance between hand and body. Then, the HMM-FNN model, which combines the hidden markov model (HMM) and the fuzzy neural network (FNN), is used for dynamic gesture recognition. This paper mainly focuses on the trainees’ common operations of equipment in virtual substation to set the custom gesture interaction sets. Based on the characteristic of the complex dynamic gesture, gesture image was decomposed into three feature sequences—hand shape change, hand position changes in the two-dimensional plane, and movement in the Z-axis direction, for feature extraction. The HMM model is respectively built according to the three sub sequences, and the FNN was connected to judge the semantics of gesture using the fuzzy reasoning. By experimental verification, the HMM-FNN model can quickly and effectively identify complicated dynamic hand gestures. Meanwhile, it has strong robustness. The recognition effect is superior to that of the simple HMM model.