Deep Learning-Based Character Gesture Recognition

Abstract

The article describes a new text input method based on gesture recognition, which enables direct physical-to-digital text input. This enables hand-free and in-air writing without any need for keyboards, mice, etc. This is done with the help of state-of-the-art deep learning methods and a single Kinect sensor. The authors were able to achieve a high-accuracy recognition rate by using any wearable device, in contrast to the existing methods, and utilizing a single sensor. Furthermore, among several existing deep learning structures, the authors determined that the best deep learning structure suitable for the character-based gesture data is the DenseNet Convolutional neural network. For instance, the training loss curve shows that DenseNet has the fastest converging curve compared to the others despite maintaining the highest accuracy rate. Our proposed method allows for the improvement of the recognition rate from 96.6% (in the existing algorithms) to 98.01% when the DenseNet structure is used despite using only a single sensor instead of multiple cameras. The use of the Kinect sensor not only reduces the number of cameras to one but also overrides the necessity for any additional hand detection algorithms. These results aid in improving the speed and the efficiency of the character-based gesture recognition. The proposed system can be used in applications that require accurate decision making, such as in operation rooms.