Dynamic Sign Language Recognition Using Gaussian Process Dynamical Models

Abstract

This paper proposes the temporal modeling of the dynamic behavior of the Colombian sign language by using a dynamic latent space captured by a Gaussian process latent variable models. The database is built from the recording of 5 signs, which are recorded 40 times each, implying a total of 200 repetitions per class. Each recording consists of a movement composed of 120 frames, where each frame is the skeleton of the subject involved. To ensure system adaptability to different test subjects, the 200 instances are performed by 4 different subjects. The time series generated for each point in the skeleton produces a large amount of information that hinder the proper classification of the signs. For this reason, a Gaussian Process Dynamical Models (GPDM) is used to model the dynamics of the sign language. Furthermore, GPDM generates points associated with a latent space for a specific behavior according to the dynamics of the sign. Within the latent space, we use three different machines for pattern classification: support vector machine (SVM), artificial neural network (ANN), and least squares for classification (LSC). Finally, some static features from the time series are extracted and classified to compare with the dynamic features. The experimental results show that by modeling the temporal behavior of the sign signals, we can recognize accurately a given sign language.