CNN-based gesture recognition using raw numerical gray-scale images of surface electromyography

Abstract

Listen

Translate article

The gesture recognition based on surface electromyography signals (sEMG) is an important human–computer interaction technology. This paper proposes a precise gesture recognition method that processes sEMG into raw numerical grayscale images and combines them with Convolutional Neural Networks (CNN) as classifiers to address challenges such as the difficulty of manual feature extraction from sEMG and low classifier recognition accuracy. Firstly, sEMG data from four channels are collected for 13 specific hand gestures from 13 subjects. Applying linear and exponential operations to sEMG voltage values in the time domain, it maps the resulting matrix to a numerical range of 0–255 and converts it into a single-layer grayscale image. Subsequently, a Convolutional Neural Network classification model (CNN4-M) is constructed, and the model is trained using these single-layer grayscale images. To assess the grayscale image method, DB1 and DB3 datasets from Ninapro were used for validation. Six classical CNNs are trained using these single-layer grayscale images, and their training results are compared to sEMG spectrograms generated using Short-Time Fourier Transform (STFT). Additionally, the gesture recognition method proposed in this paper is compared to five common machine learning methods. Results showed that the original grayscale images worked well with CNNs, outperforming STFT-generated images during validation. The gesture recognition method proposed in this paper outperforms five common machine learning methods in terms of recognition accuracy. Finally, training CNN4-M with the original grayscale images reached the highest validation accuracy, with 98.03% classification accuracy for 13 gestures and 99.95% and 98.07% on DB1 and DB3 datasets, respectively.