Kinect sensor based sign language word recognition by mutli-stream HMM

Abstract

The aim of our research is a development of the sign language recognition (SLR) system for smooth communication between hearing disability people and hearing people. Sign language is consisted of three elements; hand position, hand movement, and hand shape. The proposed method is reflected all elements. Multi-stream HMM is used for the learning and recognition processes. In the experiment, two datasets: 100 Japanese sign language words dataset and 20 Italian gesture dataset, are used for evaluating the proposed method. As a result, all three elements of hand position, hand movement, and LBP-based hand shape are efficient for SLR task.