Sign language word recognition using via-point information and correlation of they bimanual movements

Abstract

We have studied Japanese sign Language (JSL) recognition system. In our previous research, we focus on only JSL words performed in the movement of the dominant arm and proposed a recognition method using via-points extracted from the trajectory data of the dominant arm as feature points based on the minimum jerk model. In this study, in order to recognize JSL words performed in bimanual movements, we investigated an integration method of the matching result of the both arms. We classified JSL movements into three categories as part of sign language recognition system. And we used a correlation coefficient and difference of the path length between the both arm movements as a factor to classify JSL. As a result of recognition experiment, the recognition rate was 98% or more in 80 words from multiple speakers.