Isolated Sign Language Recognition with Depth Cameras

Abstract

In this paper, an approach to isolated sign language recognition with data provided by a depth camera is presented. In the introduced method, sequences of depth maps of dynamic sign language gestures are divided into smaller regions (cells). Then, statistical information is used to describe the cells. Since gesture executions have different lengths, the dynamic time warping (DTW) technique with the nearest neighbor (NN) rule is often employed for their comparison. However, due to time-consuming computations, the DTW limits the usability of the classifier. Therefore, in this paper, a selection of representative depth maps using statistics for cells is proposed. It is shown that such gesture representation can be successfully employed for isolated sign language recognition with the NN classifier using the city block distance. Furthermore, the NN rule with the DTW and the introduced statistics for cells provides superior gesture recognition performance.