Signing Exact English (SEE): Modeling and recognition

Abstract

We present effective and robust algorithms to recognize isolated signs in Signing Exact English (SEE). The sign-level recognition scheme comprises classifiers for handshape, hand movement and hand location. The SEE gesture data are acquired using CyberGlove ® and magnetic trackers. A linear decision tree with Fisher's linear discriminant (FLD) is used to classify 27 SEE handshapes. Hand movement trajectory is classified using vector quantization principal component analysis (VQPCA). Both periodic and non-periodic SEE sign gestures are recognized from isolated 3-D hand trajectories. Experiments yielded average handshape recognition accuracy of 96.1% on “unseen” signers. The average trajectory recognition rate with VQPCA for non-periodic and periodic gestures was 97.3% and 97.0%, respectively. These classifiers were combined with a hand location classifier for sign-level recognition, yielding an accuracy of 86.8% on a 28 sign SEE vocabulary.