Hand Gesture Recognition System Research Articles

An improved hand gesture recognition system using keypoints and hand bounding boxes

Hand gesture recognition is a significant problem for human–computer interaction. One form of hand gesture recognition is static hand gestures. This study developed a static hand gesture recognition system, consisting of three modules: Feature extraction Module, Processing Module, and Classification Module. The feature extraction module uses human pose estimation with a top-down method to extract not only the keypoints but also body and hand bounding boxes. After being normalized and processed in the processing module, its output will be used as the input for the classification module in which we proposed an architecture called Two-pipeline architecture. In this module, we also employ different methods to find the most suitable one for this task. Experiments were conducted on three datasets called HANDS, OUHANDS, and SHAPE. Results showed that the proposed Two-pipeline architecture with 2.5 million parameters obtained accuracy of 94%, 98%, and 94% on three datasets. In addition, the lightweight version with 0.22 million parameters also achieved accuracy of 91%, 94%, and 96%.

A Flexible Iontronic Capacitive Sensing Array for Hand Gesture Recognition Using Deep Convolutional Neural Networks.

Hand gesture recognition, one of the most popular research topics in human-machine interaction, is extensively used in visual and augmented reality, sign language translation, prosthesis control, and so on. To improve the flexibility and interactivity of wearable gesture sensing interfaces, flexible electronic systems for gesture recognition have been widely studied. However, these systems are limited in terms of wearability, stability, scalability, and robustness. Herein, we report a flexible wearable hand gesture recognition system that is based on an iontronic capacitive pressure sensing array and deep convolutional neural networks. The entire capacitive array is integrated into a flexible silicone wristband and can be comfortably and conveniently wrapped around the wrist. The pressure sensing array, which is composed of an iontronic film sandwiched between two flexible screen-printed electrode arrays, exhibits a high sensitivity (775.8 kPa-1), fast response time (65 ms), and high durability (over 6000 cycles). Image processing techniques and deep convolutional neural networks are applied for sensor signal feature extraction and hand gesture recognition. Several contexts such as intertrial test (average accuracy of 99.9%), intersession rewearing (average accuracy of 93.2%), electrode shift (average accuracy of 83.2%), and different arm positions during measurement (average accuracy of 93.1%) are evaluated.

A Hand Gesture Recognition Circuit Utilizing an Analog Voting Classifier

Electromyography is a diagnostic medical procedure used to assess the state of a muscle and its related nerves. Electromyography signals are monitored to detect neuromuscular abnormalities and diseases but can also prove useful in decoding movement-related signals. This information is vital to controlling prosthetics in a more natural way. To this end, a novel analog integrated voting classifier is proposed as a hand gesture recognition system. The voting classifiers utilize 3 separate centroid-based classifiers, each one attached to a different electromyographic electrode and a voting circuit. The main building blocks of the architecture are bump and winner-take-all circuits. To confirm the proper operation of the proposed classifier, its post-layout classification results (91.2% accuracy) are compared to a software-based implementation (93.8% accuracy) of the same voting classifier. A TSMC 90 nm CMOS process in the Cadence IC Suite was used to design and simulate the following circuits and architectures.

A virtual surgical prototype system based on gesture recognition for virtual surgical training in maxillofacial surgery

BackgroundVirtual reality (VR) technology is an ideal alternative of operation training and surgical teaching. However, virtual surgery is usually carried out using the mouse or data gloves, which affects the authenticity of virtual operation. A virtual surgery system with gesture recognition and real-time image feedback was explored to realize more authentic immersion.MethodGesture recognition technology proposed with an efficient and real-time algorithm and high fidelity was explored. The recognition of hand contour, palm and fingertip was firstly realized by hand data extraction. Then, an Support Vector Machine classifier was utilized to classify and recognize common gestures after extraction of feature recognition. The algorithm of collision detection adopted Axis Aligned Bounding Box binary tree to build hand and scalpel collision models. What’s more, nominal radius theorem (NRT) and separating axis theorem (SAT) were applied for speeding up collision detection. Based on the maxillofacial virtual surgical system we proposed before, the feasibility of integration of the above technologies in this prototype system was evaluated.ResultsTen kinds of signal static gestures were designed to test gesture recognition algorithms. The accuracy of gestures recognition is more than 80%, some of which were over 90%. The generation speed of collision detection model met the software requirements with the method of NRT and SAT. The response time of gesture] recognition was less than 40 ms, namely the speed of hand gesture recognition system was greater than 25 Hz. On the condition of integration of hand gesture recognition, typical virtual surgical procedures including grabbing a scalpel, puncture site selection, virtual puncture operation and incision were carried out with realization of real-time image feedback.ConclusionBased on the previous maxillofacial virtual surgical system that consisted of VR, triangular mesh collision detection and maxillofacial biomechanical model construction, the integration of hand gesture recognition was a feasible method to improve the interactivity and immersion of virtual surgical operation training.

A Critical Review on Hand Gesture Recognition using sEMG: Challenges, Application, Process and Techniques

Hand gesture recognition systems are gaining popularity these days due to the ease with which humans and machines can communicate. The goal of hand gesture development is to improve interactions between humans and computers for the purpose of transmitting ideas. In a typical HGR systems, the main steps followed are, data collection, pre-processing, feature extraction and classification. For every stage, a significant number of techniques are available with various other sub steps. This study gives an overview of modern hand gesture recognition techniques, its Physiological and Anatomical Background, working and challenges faced by these systems. Moreover, the role of artificial intelligence in optimizing the performance of HGR systems is also delineated in this paper. Also, the precision and accuracy of the HGR approaches gets affected by the complexity and diversity of various hand movements, therefore, the need for implementing AI based ML and DL methods keeps on rising. Keeping this in mind, the performance of various ML algorithms in recognizing the visual and sensor-based hand gestures is investigated. Moreover, the commonly utilized framework in detecting hand gestures has been explored in numerous standard datasets.

Retraction Note to: Development of hand gesture recognition system using machine learning

Retraction Note to: Development of hand gesture recognition system using machine learning

Hand Gesture Detection and Recognition

The design introduces an operation using computer vision for Hand gesture recognition. A camera records a live videotape sluice, from which a shot is taken with the help of interface. The system is trained for each type of count hand gestures (one, two, three, four, and five) at least formerly. After that a test gesture is given to it and the system tries to fete it. A exploration was carried out on a number of algorithms that could best separate a hand gesture. It was set up that the slant sum algorithm gave the loftiest delicacy rate. In the preprocessing phase, a tone- developed algorithm removes the background of each training gesture. After that the image is converted into a double image and the totalities of all slant rudiments of the picture are taken. This sum helps us in secerning and classifying different hand gestures. former systems have used data gloves or labels for input in the system. I've no similar constraints for using the system. The stoner can give hand gestures in view of the camera naturally. A fully robust hand gesture recognition system is still under heavy exploration and development; the enforced system serves as an extendible foundation for unborn work.

A Gesture Recognition Strategy Based on A-Mode Ultrasound for Identifying Known and Unknown Gestures

Human-machine interface(HMI) technology has gradually become a research hotspot with the continuous development of computer technology and Internet of Things (IOT) technology. Hand gesture recognition (HGR) as an important part of HMI technology has been widely concerned. Among the many technical routes of HGR technology, the wearable HGR technology based on A-mode ultrasound shows great application potential due to its advantages such as lightweight device, free from sensors’ constraint etc. However, in the process of processing A-mode ultrasonic signal, the amplitude of the signal at the main position may vary due to muscle fatigue and strength etc. when the same gesture is repeated some time later. In this paper, we design a method to overcome this problem and accomplish HGR in offline state by setting the threshold and use normalization method for the energy feature of ultrasonic signal according to the threshold. Three machine learning algorithms, including support vector machine (SVM), linear discriminant analysis (LDA) and Naive Bayes (NB) were used to verify the feasibility of this method, and the average recognition accuracy in the experiment reached 95.26%. Signal stability is also verified to be improved. In addition, We design a recognition strategy based on NB algorithm in this paper so that the model can identify the unknown gestures (i.e. gestures that have not been trained by the model). Five known gestures and five unknown gestures were set in the experiment, the average recognition accuracy of the known gestures is 91.2%, and the unknown gestures is 81.8%. This method can be used to optimize the user experience of HGR system.

Hand Gesture Recognition

Abstract: Hand gesture recognition system has evolved within the recent few years due to evolving of technology and mostly due to device dependent life. People are now turning towards technology for a far better life with computer and portable devices being an enormous part of it. Not only people but companies too are shifting towards technology for improved performance of their products. Keywords: Hand gesture, computer vision, deep learning, feature extraction classification, Neural network.

A Structured and Methodological Review on Vision-Based Hand Gesture Recognition System.

Researchers have recently focused their attention on vision-based hand gesture recognition. However, due to several constraints, achieving an effective vision-driven hand gesture recognition system in real time has remained a challenge. This paper aims to uncover the limitations faced in image acquisition through the use of cameras, image segmentation and tracking, feature extraction, and gesture classification stages of vision-driven hand gesture recognition in various camera orientations. This paper looked at research on vision-based hand gesture recognition systems from 2012 to 2022. Its goal is to find areas that are getting better and those that need more work. We used specific keywords to find 108 articles in well-known online databases. In this article, we put together a collection of the most notable research works related to gesture recognition. We suggest different categories for gesture recognition-related research with subcategories to create a valuable resource in this domain. We summarize and analyze the methodologies in tabular form. After comparing similar types of methodologies in the gesture recognition field, we have drawn conclusions based on our findings. Our research also looked at how well the vision-based system recognized hand gestures in terms of recognition accuracy. There is a wide variation in identification accuracy, from 68% to 97%, with the average being 86.6 percent. The limitations considered comprise multiple text and interpretations of gestures and complex non-rigid hand characteristics. In comparison to current research, this paper is unique in that it discusses all types of gesture recognition techniques.

CONTROLLING VIDEOLAN CLIENT MEDIA USING LUCAS KANADE OPTIMAL FLOW ALGORITHM AND OPENCV

In this project we've discussed a system which uses some dynamic hand gesture recognition technique to manage the media players like VLC media player. This project consist of certain modules which segments the foreground a part of the frame using detection of skin colour and approximate median technique. the popularity of hand gesture is finished by creating a call Tree, that uses certain features extracted from the segmented part. This hand gesture recognition technique introduces a replacement, natural thanks to interact with computers and is beneficial to several folks in our day-to-day life. Hand gestures are associated with the human hands in hand gesture recognition. This project is employed for controlling certain operations (Pause, Play, Volume up, Volume Down, Mute) on video player by mere hand gestures without getting in the rigmarole of pressing buttons or tapping onto the screen. this may be directly associated with our day to life like in presentations. We have considered hand gestures and their directional motion defines a gesture for the applying. during this application image retrieving is finished employing a Webcam. Some functions in VLC media players are used most often and these functions uses some predefined gestures. Shows the defined gestures in keeping with the VLC player control operation. We created a VLC Media Player Controller using Hand Gesture Recognition System to form ‘HUMAN LIFE EASY AND BETTER’. This project is implemented using two steps: (1.) Creation of Hand Gesture Recognition System: this is often done using image processing using OpenCV library. (2.) Controlling VLC Media Player using hand gestures: during this step we controlled the player using shell commands which were recalled using python commands through OS library

Long Hands gesture recognition system: 2 step gesture recognition with machine learning and geometric shape analysis

A two stage real-time hand gesture recognition system is presented. It combines a machine learning trained detection step with a colour processing contour shape validation step. The detection step is done with either Adaboost Cascades or Support Vector Machines using HOG features. The system achieves a low false positive rate and a sufficient true positive rate necessary for robust real-time performance. It performs well compared to MobileNets a state of the art Neural Network for mobile real-time applications.

A novel hand gesture detection and recognition system based on ensemble-based convolutional neural network

Nowadays, hand gesture recognition has become an alternative for human-machine interaction. It has covered a large area of applications like 3D game technology, sign language interpreting, VR (virtual reality) environment, and robotics. But detection of the hand portion has become a challenging task in computer vision and pattern recognition communities. Deep learning algorithm like convolutional neural network (CNN) architecture has become a very popular choice for classification tasks, but CNN architectures suffer from some problems like high variance during prediction, overfitting problem and also prediction errors. To overcome these problems, an ensemble of CNN-based approaches is presented in this paper. Firstly, the gesture portion is detected by using the background separation method based on binary thresholding. After that, the contour portion is extracted, and the hand region is segmented. Then, the images have been resized and fed into three individual CNN models to train them in parallel. In the last part, the output scores of CNN models are averaged to construct an optimal ensemble model for the final prediction. Two publicly available datasets (labeled as Dataset-1 and Dataset-2) containing infrared images and one self-constructed dataset have been used to validate the proposed system. Experimental results are compared with the existing state-of-the-art approaches, and it is observed that our proposed ensemble model outperforms other existing proposed methods.

Gesture Recognition System Using 24 GHz FMCW Radar Sensor Realized on Real-Time Edge Computing Platform

Affected by the global epidemic, non-contact unmanned control system provides people with safe human-computer interaction (HCI). This paper presents a radar hand gesture recognition system based on real-time edge computing platform. The system uses a low-cost 24 GHz commercial low bandwidth frequency modulated continuous wave (FMCW) radar as the detection source. For real-time gesture recognition, the background echo of radar is fitted by the exponential weighted average method which can reduce the hardware resource consumption when implementing on field programmable gate array (FPGA). A novel extracted feature named range-Doppler matrix focus (RDMF) is designed to reduce the data dimension and preserve the useful information. 3-D CNN + LSTM encoder classification framework is used to classify the features. A data set of 3200 samples for the basic 8 gestures and 1600 samples for the additional 4 gestures is designed and collected for training and testing the classification framework. The results show that the average accuracy of 3-D CNN + LSTM encoder on training and testing data set of 12 gesture classification exceeds 97.9%. The model size of this network is nearly 390 kB. In order to compress the size for recognition on edge-computing platform, a simplified framework for sending one antenna’s RDMF to the LSTM encoder is proposed. The model size is reduced to 20 kB, and the average accuracy of the simplified framework on basic 8 gestures is 95.9%. We deploy this gesture recognition framework on the FPGA platform to speed up computing.

A lightweight hand gesture recognition in complex backgrounds

Hand Gesture Recognition (HGR) is widely used in human–computer interaction due to its convenience. However, there are still some challenges in real-world scenarios, such as recognizing hand gestures in the complex backgrounds. To this end, the paper proposes a two-stage HGR system to solve the above issue. Specifically, the first stage performs accurate segmentation to segment the hand from the background. The segmentation network combines dilated residual network, atrous spatial pyramid pooling module and a simplified decoder. The segmentation network can effectively determine hand region even in challenging backgrounds. In the second stage, the double-channel Convolutional Neural Networks (CNNs) are presented to improve the recognition performance. The double-channel CNNs can learn features from the RGB input images and the segmented hand images separately. Experiment results show that the proposed method has an accuracy of 91.17% with the model size of 1.8 MB, both of which are better than other state-of-arts in hand gesture recognition. The method successfully constructed a lightweight model while keeping a high gesture recognition accuracy at final.

PowerPoint slideshow navigation control with hand gestures using Hidden Markov Model method

Gesture is the easiest and most expressive way of communication between humans and computers, especially gestures that focus on hand and facial movements. Users can use simple gestures to communicate their ideas with a computer without interacting physically. One form of communication between users and machines is in the teaching and learning process in college. One of them is the way the speakers deliver material in the classroom. Most speakers nowadays make use of projectors that project PowerPoint slides from a connected laptop. In running the presentation, the speaker needs to move a slide from one slide to the next or to the previous slide. Therefore, a hand gesture recognition system is needed so it can implement the above interactions. In this study, a PowerPoint navigation control system was built. Digital imaging techniques use a combination of methods. The YCbCr threshold method is used to detect skin color. Furthermore, the morphological method is used to refine the detection results. Then the background subtraction method is used to detect moving objects. The classification method uses the Hidden Markov Model (HMM). With 526 hand images, the result shows that the accuracy of the confusion matrix is 74.5% and the sensitivity is 76.47%. From the accuracy and sensitivity values, it can be concluded that the Hidden Markov Model method can detect gestures quite well as a PowerPoint slide navigation control.

Towards a Feasible Hand Gesture Recognition System as Sterile Non-contact Interface in the Operating Room with 3D Convolutional Neural Network

Operating surgeons are constrained when interacting with computer systems as they traditionally utilize hand-held devices such as keyboard and mouse. Studies have previously proposed and shown the use of hand gestures is an efficient, touchless way of interfacing with such systems to maintain a sterile field. In this paper, we propose a Deep Computer Vision-based Hand Gesture Recognition framework to facilitate the interaction. We trained a 3D Convolutional Neural Network with a very large scale dataset to classify hand gestures robustly. This network became the core component of a prototype application requiring intraoperative navigation of medical images of a patient. Usability evaluation with surgeons demonstrates the application would work and a hand gesture lexicon that is germane to Medical Image Navigation was defined. By completing one cycle of usability engineering, we prove the feasibility of using the proposed framework inside the Operating Room.

Touchless Control of Heavy Equipment Using Low-Cost Hand Gesture Recognition

Human-machine interaction using remote hand gestures is becoming increasingly prevalent across various industries. However, their potential application to heavy construction equipment is often overlooked. This article presents a robust and inexpensive hand gesture recognition system that was implemented and tested on a robotic 1-tonne wheel loader. The system uses an RGB camera paired with a laptop to process, in real time, hand gestures to control the loader. We first design four unique gestures for controlling the loader and then collect 26,000 images to train and test a neural network for hand gesture recognition. Our system uses robust landmark detection using an off-the-shelf system prior to gesture recognition. We successfully controlled the loader to excavate in a rock pile by using the proposed hand gesture recognition system.

Using the virtual data-driven measurement to support the prototyping of hand gesture recognition interface with distance sensor

The performance of a fixed-sensor-based hand gesture recognition system is typically influenced by the position and number of sensors. The traditional development approach to hand gesture recognition systems follows a process of sensor pre-deployment, data collection, and model training, which is highly time-consuming and expensive to calculate how many sensors to place where, and the system has low secondary development flexibility. In this paper, we present a new development flow to assist in prototyping distance sensor-based gesture recognition interfaces. The designed system was able to simulate the position and number of sensors to recognize gestures. Using a reconstructed hand motion, the virtual distance sensor generated simulated signals and trained a convolutional neural network model. In a real-world setting, the sensor system only needed to be configured by transfer learning to recognize gestures at the same sensor layout. The proposed method was able to indicate sensor configuration and the trained classifier via virtual distance data, which can effectively reduce the development cost. We evaluated two prototype interfaces of the proposed method using distance sensors and demonstrated that the system effectively provided deployment recommendations, and models trained using virtual measurement data could effectively recognize real gestures.

CNN-LSTM Hybrid Real-Time IoT-Based Cognitive Approaches for ISLR with WebRTC: Auditory Impaired Assistive Technology.

In the era of modern technology, people may readily communicate through facial expressions, body language, and other means. As the use of the Internet evolves, it may be a boon to the medical fields. Recently, the Internet of Medical Things (IoMT) has provided a broader platform to handle difficulties linked to healthcare, including people's listening and hearing impairment. Although there are many translators that exist to help people of various linguistic backgrounds communicate more effectively. Using kinesics linguistics, one may assess or comprehend the communications of auditory and hearing-impaired persons who are standing next to each other. When looking at the present COVID-19 scenario, individuals are still linked in some way via online platforms; however, persons with disabilities have communication challenges with online platforms. The work provided in this research serves as a communication bridge inside the challenged community and the rest of the globe. The proposed work for Indian Sign Linguistic Recognition (ISLR) uses three-dimensional convolutional neural networks (3D-CNNs) and long short-term memory (LSTM) technique for analysis. A conventional hand gesture recognition system involves identifying the hand and its location or orientation, extracting certain essential features and applying an appropriate machine learning algorithm to recognise the completed action. In the calling interface of the web application, WebRTC has been implemented. A teleprompting technology is also used in the web app, which transforms sign language into audible sound. The proposed web app's average recognition rate is 97.21%.