Large-Vocabulary Continuous Sign Language Recognition Based on Transition-Movement Models

Abstract

The major challenges that sign language recognition (SLR) now faces are developing methods that solve large-vocabulary continuous sign problems. In this paper, transition-movement models (TMMs) are proposed to handle transition parts between two adjacent signs in large-vocabulary continuous SLR. For tackling mass transition movements arisen from a large vocabulary size, a temporal clustering algorithm improved from k-means by using dynamic time warping as its distance measure is proposed to dynamically cluster them; then, an iterative segmentation algorithm for automatically segmenting transition parts from continuous sentences and training these TMMs through a bootstrap process is presented. The clustered TMMs due to their excellent generalization are very suitable for large-vocabulary continuous SLR. Lastly, TMMs together with sign models are viewed as candidates of the Viterbi search algorithm for recognizing continuous sign language. Experiments demonstrate that continuous SLR based on TMMs has good performance over a large vocabulary of 5113 Chinese signs and obtains an average accuracy of 91.9%