A Fully Embedded Adaptive Real-Time Hand Gesture Classifier Leveraging HD-sEMG and Deep Learning

Abstract

This paper presents a real-time fine gesture recognition system for multi-articulating hand prosthesis control, using an embedded convolutional neural network (CNN) to classify hand-muscle contractions sensed at the forearm. The sensor consists in a custom non-intrusive, compact, and easy-to-install 32-channel high-density surface electromyography (HDsEMG) electrode array, built on a flexible printed circuit board (PCB) to allow wrapping around the forearm. The sensor provides a low-noise digitization interface with wireless data transmission through an industrial, scientific and medical (ISM) radio link. An original frequency-time-space cross-domain preprocessing method is proposed to enhance gesture-specific data homogeneity and generate reliable muscle activation maps, leading to 98.15% accuracy when using a majority vote over 5 subsequent inferences by the proposed CNN. The obtained real-time gesture recognition, within 100 to 200ms, and CNN properties show reliable and promising results to improve on the state-of-the-art of commercial hand prostheses. Moreover, edge computing using a specialized embedded artificial intelligence (AI) platform ensures reliable, secure and low latency real-time operation as well as quick and easy access to training, fine-tuning and calibration of the neural network. Co-design of the signal processing, AI algorithms and sensing hardware ensures a reliable and power-efficient embedded gesture recognition system.