A wearable sign language translation device utilizing silicone-hydrogel hybrid triboelectric sensor arrays and machine learning

Abstract

Sign language is a crucial communication tool for the hearing impaired to interact with the outside world. However, the low prevalence of sign language leads to significant communication barriers between the hearing impaired and others. These barriers can be alleviated by using electronic sign language translation devices, but such devices typically face challenges related to their bulkiness, rigidity, and high cost. Herein, we present a cost-effective, flexible, and wearable device designed for the interpretation of sign language, leveraging machine learning algorithms to transform sign language movements into spoken language accurately. Our device is equipped with flexible and stretchable triboelectric sensor (FS-TS) arrays and a printed circuit board for efficient signal processing and wireless transmission. These FS-TSs are constructed from cost-effective materials including silicone, hydrogel, and fluorinated ethylene propylene (FEP) powders, which, while affordable, ensure rapid response and heightened sensitivity. By analyzing 1,000 sets of sign language gestures with machine learning, our system has achieved an impressive recognition accuracy of 98.5%. This achievement underscores the potential of our system as an economically viable and scalable solution for enhancing sign language recognition within the wearable electronics domain.