Character Input System using Fingertip Detection with Kinect Sensor

Abstract

The sign language and the finger alphabet are used for instrumentalizing communication with the deaf and hard of hearing people. Therefore, a character input system using hand gestures has already been extensively investigated. Previous research has recognized sign language, the finger alphabet and hand writing in this regard; however each has some problems. The problems are character recognition time, and a lot of required knowledge about sign language and the finger alphabet. So, many people cannot use sign language or the finger alphabet. The purpose of this paper is to make a character input system that can be used by anyone. This system facilitates communication among hearing-impaired persons and users who cannot use sign language or the finger alphabet without the use of a keyboard, mouse and body-worn computer. This system is a novel approach. The fingertip is detected by the coordinate of the hand position from the skeleton information. Moreover, the flick input can be performed in the air by using the coordinates of the head, both shoulders, and both hands' position. As a result, our system can be used by anyone with only a brief description; further smooth character input is possible using the Kinect sensor and flick input. In addition, the system supports communication not only with Japanese, but also English and numbers. It is expected that our system will enlarge the range of communication.