Analysis of Sensors in the Classification of the Brazilian Sign Language

Abstract

The use of hands during communication is generally a natural way of communication and allows the transmission of messages. However, in order to communicate with deaf people using sign language, it is necessary to know the rules of sign language. In this way, tools and technologies for communication between deaf people and listeners are very important to improve the social inclusion of the deaf community. The wearable hand devices have been used to identify sign language gestures. In this regard, the purpose of this paper is to analyze the contribution of these sensors, individually and in groups, in the classification of characters of Brazilian sign language alphabet. The wearable hand device used is composed of five flex sensors, two contact sensors, and a three-axis inertial sensor (accelerometer and gyroscope). For the dataset, five volunteers executed 26 alphabet characters. For the classification, each gesture window was divided into three parts: construction period, gesture period and gesture relaxation period. Thus, 28 gesture patterns were considered for classification. The used classifier was the Multilayer Perceptron Neural Network (MLP-NN). In the individual sensor analysis, the flex-sensors obtained the highest accuracy rate (79.0%), followed by the accelerometer (41.7%), the gyroscope (32.1%), the contact sensor 2 (7.1%), and the contact sensor 1 (6.9%). In the analysis of the sensors in groups, the set of all sensors was the one that obtained the highest accuracy rate (96.1%).