Gesture Recognition through Mechanomyogram Signals: An Adaptive Framework for Arm Posture Variability.

Abstract

In hand gesture recognition, classifying gestures across multiple arm postures is challenging due to the dynamic nature of muscle fibers and the need to capture muscle activity through electrical connections with the skin. This paper presents a gesture recognition architecture addressing the arm posture challenges using an unsupervised domain adaptation technique and a wearable mechanomyogram (MMG) device that does not require electrical contact with the skin. To deal with the transient characteristics of muscle activities caused by changing arm posture, Continuous Wavelet Transform (CWT) combined with Domain-Adversarial Convolutional Neural Networks (DACNN) were used to extract MMG features and classify hand gestures. DACNN was compared with supervised trained classifiers and shown to achieve consistent improvement in classification accuracies over multiple arm postures. With less than 5 minutes of setup time to record 20 examples per gesture in each arm posture, the developed method achieved an average prediction accuracy of 87.43% for classifying 5 hand gestures in the same arm posture and 64.29% across 10 different arm postures. When further expanding the MMG segmentation window from 200 ms to 600 ms to extract greater discriminatory information at the expense of longer response time, the intraposture and inter-posture accuracies increased to 92.32% and 71.75%. The findings demonstrate the capability of the proposed method to improve generalization throughout dynamic changes caused by arm postures during non-laboratory usages and the potential of MMG to be an alternative sensor with comparable performance to the widely used electromyogram (EMG) gesture recognition systems.