Wearable multifunctional organohydrogel-based electronic skin for sign language recognition under complex environments

Abstract

Language barrier is the main cause of disagreement. Sign language, which is a common language in all the worldwide language families, is difficult to be entirely popularized due to the high cost of learning as well as the technical barrier in real-time translation. To solve these problems, here, we constructed a wearable organohydrogel-based electronic skin (e-skin) with fast self-healing, strong adhesion, extraordinary anti-freezing and moisturizing properties for sign language recognition under complex environments. The e-skin was obtained by using an acrylic network as the main body, aluminum (III) and bayberry tannin as the crosslinking agent, water/ethylene glycol as the solvent system, and a polyvinyl alcohol network to optimize the network performance. Using this e-skin, a smart glove was further built, which could carry out the large-scale data collection of common gestures and sign languages. With the help of the deep learning method, specific recognition and translation for various gestures and sign languages could be achieved. The accuracy was 93.5%, showing the ultra-high classification accuracy of a sign language interpreter. In short, by integrating multiple characteristics and combining deep learning technology with hydrogel materials, the e-skin achieved an important breakthrough in human-computer interaction and artificial intelligence, and provided a feasible strategy for solving the dilemma of mutual exclusion between flexible electronic devices and human bodies.