Online cross session electromyographic hand gesture recognition using deep learning and transfer learning

Abstract

In recent years, hand gesture recognition in human-computer interfaces is usually based on surface electromyography because the signals are non-intrusive and are not affected by the variations of light, position, and orientation of the hand. Deep learning algorithms have become increasingly more prominent in gesture recognition for the ability to automatically learn features from large amounts of data. However, delicate and complicated network structures brought by deep learning, which are elaborately designed for cross session tasks, need more computing time to be trained and tested, which can hardly be applied to the online system. In this study, an online electromyographic hand gesture recognition method using deep learning and transfer learning is proposed. The deep learning model includes a feature extractor, a label classifier, and a gesture predictor. The feature extractor is based on the temporal convolutional network, which is designed to learn high-level discriminant features from the input signals. The label classifier includes three fully connected layers, designed to classify hand gesture labels using the feature vector which is produced by the feature extractor. The gesture predictor uses a threshold voting algorithm to predict the gesture, used at the stage of testing to perform the online recognition. Transfer learning technique is used to transfer model parameters from one pre-trained model, which costs less time and can be applied for online applications. The proposed model is verified on both the Myo dataset and the public NinaPro database. The proposed transfer learning scheme is shown to systematically and significantly enhance the performance of the proposed model on the two datasets, only using no more than three sessions to retrain the label predictor can achieve the accuracy of more than 90% of that obtained though the normal training of the whole parts of the model using full training sessions.