An Optical Flow Based Approach to Detect Movement Epenthesis in Continuous Fingerspelling of Sign Language

Abstract

In this paper, the movement epenthesis problem in continuous fingerspelling is addressed. Movement epenthesis caused due to unwanted but unavoidable hand movement in between two sign gestures in continuous signing is one of the major problems in automatic sign language recognition. A novel method based on calculating the 2-norm values of the magnitude matrices of optical flow has been proposed in this paper to detect the movement epenthesis containing video frames. We used Horn-Schunck method to compute the optical flow and estimate the speed of hands in the the continuous fingerspelling videos. The 2-norm values of the magnitude matrix of optical flow provides a discriminative feature to distinguish of movement epenthesis frames from sign frames and hence mean of the 2-norm values is used as the threshold value to detect the movement epenthesis frames in a gesture video. We tested our method on continuous fingerspelling videos of Indian sign language. Experimental results show that the performance of our proposed method is 100% accurate in detecting the movement epenthesis frames in continuous fingerspelling.