A Wearable Gesture Recognition System With Ultrahigh Accuracy and Robustness Enabled by the Synergy of Multiple Fabric Sensing Devices

Abstract

Recently, numerous wearable devices for gesture recognition have been reported, however, most of them are fabricated by assembling the separately-designed sensing layers with the substrate through linking agents, which will lead to the shifting or detaching of the sensing layers after long-duration use. Subsequently, over-reliance on multiple filtering algorithms to smooth the collected datasets becomes very popular, unavoidably resulting in the loss of data features and overfitting. In our point of view, it is much more crucial and effective for gesture recognition performance to reinforce the stability of the sensing layers. Therefore, in this work, we designed a type of Integrated MXene/Polyurethane (I-M/PU) sensor based on textile fabrics and further prepared a data glove, a smart wristband and a smart elbow pad. A wearable system for gesture recognition with ultra-high sensitivity to strain, as well as excellent stability and durability is successfully realized by the synergy of those three devices. The collected original data by the integrated sensors was steady with specific features, making the simplification of the classification model feasible. Finally, 19 static gestures (including 9 complex gestures) with the recognition accuracy of 100% in 6998 data samples have been impressively achieved by the simple Multilayer Perceptron (MLP) algorithm, which is the state-of-art best result as we know in the gesture recognition reports without any assistant computer vision algorithms, demonstrating our solution with a promising prospect in Virtual Reality (VR) and Human-Machine Interaction (HMI) fields.