Gesture recognition using dual-stream CNN based on fusion of sEMG energy kernel phase portrait and IMU amplitude image

Abstract

Since the surface electromyography signal (sEMG) is a weak and unstable signal, human action recognition based on sEMG time domain or frequency domain feature extraction has poor stability and low recognition rate. Based on modeling analysis of sEMG, the dual-stream convolutional neural network (CNN) is built to perform feature fusion of sEMG energy kernel phase portrait and inertial measurement unit(IMU) data for gesture recognition in this paper. Firstly, methods of matrix counting method and time window amplitude are utilized to represent the sEMG and IMU data as images. Secondly, the moving average method is presented to filter the images. Thirdly, the dual-stream CNN is used to perform feature extraction, fusion and classification of sEMG and IMU images respectively. Finally, the experiments yield an average recognition accuracy of 95.78% for 6 gestures performed by 5 subjects, which demonstrates that the proposed method has better performance for human gestures recognition.