Machine Learning-Enabled Intelligent Gesture Recognition and Communication System Using Printed Strain Sensors.

Abstract

Gesture contains abundant and complicated information in daily life; as a consequence, gesture recognition attracts a wide range of application prospects and academic values as an important way of achieving human-machine interactions (HMIs). Here, we report an intelligent system consisting of a smart glove made by printed CNT-graphene/PDMS strain sensors. The smart glove shows excellent fitness, comfort, and lightness for human hands. Inspired by machine learning strategies, several objects and gestures can be well classified and implemented by a customized artificial neural network. Several data sets of different sign language gestures and object-grabbing gestures were established, and the result shows that the intelligent system can achieve an average accuracy of 97% and up to 99.4% for a number of gesture groups. Moreover, a robot hand is connected to this system, which is able to react to the motion of human hands with certain gestures where simple sign communication is achieved. These features provide a feasible practical application scheme for gesture recognition in HMIs.