Abstract
Sign Language Recognition (SLR) is an increasingly popular research topic due to its extensive potential applications, such as education, healthcare, emergency response, and social interaction. Sign language is a complex and dynamic language comprising hand gestures, facial expressions, and body motions. This high level of variability poses a significant obstacle for SLR tasks, which must accurately identify and respond to numerous gestures. To address these challenges, an end-to-end skeleton-based multi-feature multi-stream multi-level information sharing network (three multi information sharing network (TMS-Net)) is proposed. Specifically, in order to input more rich information to TMS-Net, we use joint feature pair with global features, bone feature pair with local features, and angle feature pair with scale invariance. In terms of network structure, to efficiently extract multiple features from inputs, we build a multi-stream structure and design a multi-level information sharing mechanism based on this structure to ensure the full utilization of skeleton feature information. From the experiment results of the WLASL-2000 dataset (56.4%), AUTSL dataset (96.62%) and MSASL(65.13%), TMS-Net surpass the state-of-the-art (SOTA) methods with single modality as input. In addition, a SLR-based human–robot interaction (HRI) experiment using our proposed TMS-Net is conducted, which proves the practical performance of the TMS-Net.
Talk to us
Join us for a 30 min session where you can share your feedback and ask us any queries you have
Disclaimer: All third-party content on this website/platform is and will remain the property of their respective owners and is provided on "as is" basis without any warranties, express or implied. Use of third-party content does not indicate any affiliation, sponsorship with or endorsement by them. Any references to third-party content is to identify the corresponding services and shall be considered fair use under The CopyrightLaw.