Sign language recognition through Leap Motion controller and input prediction algorithm

Abstract

The sign recognition systems are aimed to help deaf people communicate with society. In this paper we proposed our own concept of sign language recognition, which is based on a co-operative deep learning neural network, a text input prediction algorithm and a feedback from the user. We have pointed out the complexity of the Russian sign language and conceived the fingerspelling recognition. The method utilizes the natural properties of fingerspelling in order to increase the accuracy and recognition performance by predicting the ongoing letter. We also provide a detailed review of data acquisition in the related works. From a hardware perspective, we suggest using Leap Motion controller.