A Chinese sign language recognition system based on SOFM/SRN/HMM

Abstract

In sign language recognition (SLR), the major challenges now are developing methods that solve signer-independent continuous sign problems. In this paper, SOFM/HMM is first presented for modeling signer-independent isolated signs. The proposed method uses the self-organizing feature maps (SOFM) as different signers' feature extractor for continuous hidden Markov models (HMM) so as to transform input signs into significant and low-dimensional representations that can be well modeled by the emission probabilities of HMM. Based on these isolated sign models, a SOFM/SRN/HMM model is then proposed for signer-independent continuous SLR. This model applies the improved simple recurrent network (SRN) to segment continuous sign language in terms of transformed SOFM representations, and the outputs of SRN are taken as the HMM states in which the lattice Viterbi algorithm is employed to search the best matched word sequence. Experimental results demonstrate that the proposed system has better performance compared with conventional HMM system and obtains a word recognition rate of 82.9% over a 5113-sign vocabulary and an accuracy of 86.3% for signer-independent continuous SLR.