Real-Time Hand Gesture Recognition Based on Deep Learning YOLOv3 Model

Abdullah Mujahid,Mazhar Javed Awan,Karrar Hameed Abdulkareem,Rytis Maskeliūnas,Robertas Damaševičius,Awais Yasin,Mazin Abed Mohammed

doi:10.3390/app11094164

Abstract

Using gestures can help people with certain disabilities in communicating with other people. This paper proposes a lightweight model based on YOLO (You Only Look Once) v3 and DarkNet-53 convolutional neural networks for gesture recognition without additional preprocessing, image filtering, and enhancement of images. The proposed model achieved high accuracy even in a complex environment, and it successfully detected gestures even in low-resolution picture mode. The proposed model was evaluated on a labeled dataset of hand gestures in both Pascal VOC and YOLO format. We achieved better results by extracting features from the hand and recognized hand gestures of our proposed YOLOv3 based model with accuracy, precision, recall, and an F-1 score of 97.68, 94.88, 98.66, and 96.70%, respectively. Further, we compared our model with Single Shot Detector (SSD) and Visual Geometry Group (VGG16), which achieved an accuracy between 82 and 85%. The trained model can be used for real-time detection, both for static hand images and dynamic gestures recorded on a video.

Highlights

The interaction between humans and computers has increased widely, while the domain is witnessing continuous development, with new methods derived and techniques discovered
By using YOLOv3, our hand gesture recognition system has achieved a high accuracy even in a complex environment, and it successfully detected gestures even in low-resolution picture mode; The trained model can be used for real-time detection, it can be used for static hand images, and it can detect gestures from video feed
Output of the application that box performs real-time hand it wasAs leftyou intentionally

Summary

Introduction

The interaction between humans and computers has increased widely, while the domain is witnessing continuous development, with new methods derived and techniques discovered. Hand gesture recognition is one of the most advanced domains in which computer vision and artificial intelligence has helped to improve communication with deaf people and to support gesture-based signaling systems [1,2]. The use of hand gestures in different software applications has contributed towards improving computer and human interaction [13]. The progress of the gesture recognition systems plays a vital role in the development of computer and human interaction, and the use of hand gestures in various domains is growing more frequent. The recent development of hand gesture recognition in different sectors has grabbed the attention of industry, too, for human-robot interaction in manufacturing [21,22], and control of autonomous cars [23]

Objectives

Methods

Results

Discussion

Conclusion