Human-Computer Interaction Based on Hand Gestures Using RGB-D Sensors

José Palacios,Sergio Llorente,Eduardo Montijano,Carlos Sagüés

doi:10.3390/s130911842

José Palacios, Sergio Llorente + Show 2 more

Open Access

https://doi.org/10.3390/s130911842

Copy DOI

Abstract

In this paper we present a new method for hand gesture recognition based on an RGB-D sensor. The proposed approach takes advantage of depth information to cope with the most common problems of traditional video-based hand segmentation methods: cluttered backgrounds and occlusions. The algorithm also uses colour and semantic information to accurately identify any number of hands present in the image. Ten different static hand gestures are recognised, including all different combinations of spread fingers. Additionally, movements of an open hand are followed and 6 dynamic gestures are identified. The main advantage of our approach is the freedom of the user's hands to be at any position of the image without the need of wearing any specific clothing or additional devices. Besides, the whole method can be executed without any initial training or calibration. Experiments carried out with different users and in different environments prove the accuracy and robustness of the method which, additionally, can be run in real-time.

Highlights

In recent years, hand gesture recognition is gaining great importance in human-computer interaction (HCI) and human-robot interaction (HRI)
With the information of the hand, we present a new gesture recognition approach based on a feature-based decision tree
To test the efficiency and the robustness of our static gesture recognition system, we carried out a series of experiments

Summary

Introduction

Hand gesture recognition is gaining great importance in human-computer interaction (HCI) and human-robot interaction (HRI). Different approaches have appeared making use of different sensors and devices. Hand wearable devices such as sensor gloves [1,2] have been used they are usually expensive and user intrusive. Other less intrusive wireless devices like the Wii controller [3]. Sensing rings [4] have appeared to overcome these drawbacks. Cameras and computer vision have proved to be useful tools for this task [5]. Other contact-free sensors have emerged lately [6]

Objectives

Results

Conclusion