Real-time Gesture Recognition System Research Articles

A Novel Control System for a Laptop with Gestures Recognition

An important lesson learned during the pandemic is that the devices utilized by many people (vending machines, elevators, etc.) can spread the virus and take advantage of contactless programs that can be enabled through gesture recognition. It's a major enabler of smart living and human-computer interaction. It is widely used in controlling equipments. This research presents a novel vision-based real-time gesture recognition system for controlling a laptop. It's a comprehensive system that can do all the required functions needed to remotely control a laptop through hand gesture recognition rather than employing a mouse or keyboard. This system allows changing the position of the mouse point, performing left-click, right-click, double-click, and scrolling up and down. It also provides all the keyboard shortcuts, Print Screen (PrtScr), and Backspace. Furthermore, it allows shutting down as well as creating files and folders. Additionally, it allows playing, muting, pausing media playback, reducing/increasing the brightness, and turning the volume down/up. Moreover, it allows advancing to the next and previous page of the slideshow, as well as zooming in/out. Finally, the system allows gestures to open the most frequently used applications based on the user's choice. In comparison to the previous work, this system is considered unique in that it performs all the functions needed to remotely control the laptop. Previous systems only offer a subset of the tools provided by this proposed system Additionally, it is unique because the user operates all functions solely through gestures, without the need for additional methods.

Handwave Harmony: Transforminggestures into Words

Our project introduces a real-time gesture recognition system using ResNet50 and OpenCV. This innovative technology offers precise interpretation of sign movements for individuals with hearing difficulties. This represents a form of interpersonal connection where individuals convey understanding, empathy, or agreement through the subtle movements of their hands. The system is trained on a diverse dataset for flexibility and utilizes ResNet50's capabilities while integrating OpenCV for reduced latency, ensuring effective communication. Rigorous testing validates its accuracy, with user input fosters iterative learning. The study highlights the Importance of continuous enhancement and collaboration with the deaf and hard-of-hearing community for greater inclusiveness.

Application of AI-based real-time gesture recognition and embedded system in the design of english major teaching

Application of AI-based real-time gesture recognition and embedded system in the design of english major teaching

SonicASL

We propose SonicASL, a real-time gesture recognition system that can recognize sign language gestures on the fly, leveraging front-facing microphones and speakers added to commodity earphones worn by someone facing the person making the gestures. In a user study (N=8), we evaluate the recognition performance of various sign language gestures at both the word and sentence levels. Given 42 frequently used individual words and 30 meaningful sentences, SonicASL can achieve an accuracy of 93.8% and 90.6% for word-level and sentence-level recognition, respectively. The proposed system is tested in two real-world scenarios: indoor (apartment, office, and corridor) and outdoor (sidewalk) environments with pedestrians walking nearby. The results show that our system can provide users with an effective gesture recognition tool with high reliability against environmental factors such as ambient noises and nearby pedestrians.

A 137-μW 1.78-mm2 30-Frames/s Real-Time Gesture Recognition SoC for Smart Devices

Gesture recognition has increasingly become one of the most popular human-machine interaction techniques for smart devices. Existing gesture recognition systems suffer from either excessive power consumption or large size, limiting their applications for ultralow-power wearable devices. This article presents an accurate area-efficient and low-power real-time gesture recognition system for smart wearable devices. The proposed work utilizes an accurate peak-based gesture classification engine with less memory and a low-resolution and low-power on-chip image sensor for achieving high area efficiency and low power. In addition, the feature extraction architecture removes fixed-pattern noises from the low-power on-chip image sensor for accuracy improvement and employs parallelism for recognition speed enhancement. Thus, the proposed system accomplishes accurate real-time gesture recognition for eight motion hand gestures with an average recognition accuracy of 90.6% and latency of 4.228 ms. Measurement results of a test chip fabricated in 65-nm CMOS demonstrate that the proposed system consumes 137.0 μW at 30 frames/s while occupying only 1.78 mm <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">2</sup> , which achieves the lowest power and smallest area among the recently reported gesture recognition systems.

Deep Learning Based Real-Time Recognition of Dynamic Finger Gestures Using a Data Glove

In this article, a real-time dynamic finger gesture recognition using a soft sensor embedded data glove is presented, which measures the metacarpophalangeal (MCP) and proximal interphalangeal (PIP) joint angles of five fingers. In the gesture recognition field, a challenging problem is that of separating meaningful dynamic gestures from a continuous data stream. Unconscious hand motions or sudden tremors, which can easily lead to segmentation ambiguity, makes this problem difficult. Furthermore, the hand shapes and speeds of users differ when performing the same dynamic gesture, and even those made by one user often vary. To solve the problem of separating meaningful dynamic gestures, we propose a deep learning-based gesture spotting algorithm that detects the start/end of a gesture sequence in a continuous data stream. The gesture spotting algorithm takes window data and estimates a scalar value named gesture progress sequence (GPS). GPS is a quantity that represents gesture progress. Moreover, to solve the gesture variation problem, we propose a sequence simplification algorithm and a deep learning-based gesture recognition algorithm. The proposed three algorithms (gesture spotting algorithm, sequence simplification algorithm, and gesture recognition algorithm) are unified into the real-time gesture recognition system and the system was tested with 11 dynamic finger gestures in real-time. The proposed system took only 6 ms to estimate a GPS and no more than 12 ms to recognize the completed gesture in real-time.

Real Time Gesture Recognition System to Simulate Guitar Chords

Real Time Gesture Recognition System to Simulate Guitar Chords

A real-time gesture recognition system using near-infrared imagery.

Visual hand gesture recognition systems are promising technologies for Human Computer Interaction, as they allow a more immersive and intuitive interaction. Most of these systems are based on the analysis of skeleton information, which is in turn inferred from color, depth, or near-infrared imagery. However, the robust extraction of skeleton information from images is only possible for a subset of hand poses, which restricts the range of gestures that can be recognized. In this paper, a real-time hand gesture recognition system based on a near-infrared device is presented, which directly analyzes the infrared imagery to infer static and dynamic gestures, without using skeleton information. Thus, a much wider range of hand gestures can be recognized in comparison with skeleton-based approaches. To validate the proposed system, a new dataset of near-infrared imagery has been created, from which good results that outperform other state-of-the-art strategies have been obtained.

Vision Based Real Time Gesture Recognition

With the advent of image processing and computer vision techniques there is a huge advancement going on in the field of bio-metric analysis. Bio-metric analysis is the measurement and statistical analysis of people’s unique physical and behavioral characteristics. A wide range of applications ranging from face recognition to finger print scanning are being developed. One of the applications of bio-metrics is in gesture recognition for human-machine communication, since it leads to a more intuitive interaction and control. This paper describes a vision based real-time gesture recognition system, designed for operating in an automotive environment. We have implemented a system in which the user can change the direction of movement of a bot using gestures. These gesture are converted to commands using image processing algorithms. Commands are transmitted wireless to a robot to move it in a specific direction.

Multi-objective optimization for hand posture recognition

Hand gestures are an intuitive way for humans to interact with computers. They are becoming increasingly popular in several applications, such as smart houses, games, vehicle infotainment systems, kitchens and operating theaters. An effective human–computer interaction system should aim at both good recognition accuracy and speed. This paper proposes a new approach for static hand gesture recognition. A benchmark database with 36 gestures is used, containing variations in scale, illumination and rotation. Several common image descriptors, such as Fourier, Zernike moments, pseudo-Zernike moments, Hu moments, complex moments and Gabor features are comprehensively compared in terms of their respective accuracy and speed. Gesture recognition is undertaken by a multilayer perceptron which has a flexible structure and fast recognition. In order to achieve improved accuracy and minimize computational cost, both the feature vector and the neural network are tuned by a multi-objective evolutionary algorithm based on the Nondominated Sorting Genetic Algorithm II (NSGA-II). The proposed method is compared with state-of-the-art methods. A real-time gesture recognition system based on the proposed descriptor is constructed and evaluated. Experimental results show a good recognition rate, using a descriptor with low computational cost and reduced size.

Albanian Sign Language (AlbSL) Number Recognition from Both Hand’s Gestures Acquired by Kinect Sensors

Albanian Sign Language (AlbSL) is relatively new and until now there doesn't exist a system that is able to recognize Albanian signs by using natural user interfaces (NUI). The aim of this paper is to present a real-time gesture recognition system that is able to automatically recognize number signs for Albanian Sign Language, captured from signer's both hands. Kinect device is used to obtain data streams. Every pixel generated from Kinect device contains depth data information which is used to construct a depth map. Hands segmentation process is performed by applying a threshold constant to depth map. In order to differentiate signer's hands a K-means clustering algorithm is applied to partition pixels into two groups corresponding to each signer's hands. Centroid distance function is calculated in each hand after extracting hand's contour pixels. Fourier descriptors, derived form centroid distance is used as a hand shape representation. For each number gesture there are 15 Fourier descriptors coefficients generated which represent uniquely that gesture. Every input data is compared against training data set by calculating Euclidean distance, using Fourier coefficients. Sign with the lowest Euclidean distance is considered as a match. The system is able to recognize number signs captured from one hand or both hands. When both signer's hands are used, some of the methodology processes are executed in parallel in order to improve the overall performance. The proposed system achieves an accuracy of 91% and is able to process 55 frames per second.

Controlling Mouse using Hand Gesture Recognition

Humans communicate with one another not only through their vocal abilities but also through the gestures that they make. A gesture can go a great way in putting through our point and making the other person understand us and computers are no different in this and can be controlled and made to respond to hand gestures using a gesture recognition system. The primary goal of a gesture recognition system is to identify specific human gesture and use the gesture for controlling the device in the manner specified for that gesture in the gesture recognition system. By implementing real time gesture recognition system, a user can control a computer by simply doing gestures in front of the web cam which is attached with the laptop/computer. And to implement this system, we have used real-time hand tracking algorithm, extraction algorithm and feature extraction. General Terms Gesture Recognition, Gesture Recognition System (GRS), Hand Tracking, Feature Extraction, Background Subtraction.

Computer Vision-Based Real-Time 3D Gesture Recognition Using Depth Image

Gesture is one of the fundamental ways of human machine natural interaction. To understand gesture, the system should be able to interpret 3D movements of human. This paper presents a computer vision-based real-time 3D gesture recognition system using depth image which tracks 3D joint position of head, neck, shoulder, arms, hands and legs. This tracking is done by Kinect motion sensor with OpenNI API and 3D motion gesture is recognized using the movement trajectory of those joints. User to Kinect sensor distance is adapted using proposed center of gravity (COG) correction method and 3D joint position is normalized using proposed joint position normalization method. For gesture learning and recognition, data mining classification algorithms such as Naive Bayes and neural network is used. The system is trained to recognize 12 gestures used by umpires in a cricket match. It is trained and tested using about 2000 training instances for 12 gesture of 15 persons. The system is tested using 5-fold cross validation method and achieved 98.11% accuracy with neural network and 88.84% accuracy with Naive Bayes classification method.

Real Time Gesture Recognition System for Interaction in Dynamic Environment

Human Computer Interaction techniques have become a bottleneck in the effective utilization of the available information flow. The development of user interfaces influences the changes in the Human-Computer Interaction (HCI). Human hand gestures have been a mode of non verbal interaction widely used. Naturalistic and intuitiveness of the hand gesture has been a great motivating factor for the researchers in the area of HCI to put their efforts to research and develop the more promising means of interaction between human and computers. This paper designs a system for gestural interaction between a user and a computer in dynamic environment. The gesture recognition system uses image processing techniques for detection, segmentation, tracking and recognition of hand gestures for converting it to a meaningful command. The interface being proposed here can be substantially applied towards different applications like image browser, games etc.

A smart camera with gesture recognition and SLAM capabilities for mobile robots

This paper presents a smart camera for a real-time gesture recognition system. The smart camera has been designed and implemented as a system-on-a-chip (SoC), using reconfigurable computing technology. In this system the gesture images are captured by means of a CMOS digital camera. After some pre-processing steps, those are sent to a fault tolerant module (FTM) for the actual recognition process. The FTM implements a RAM-based neural network, using three knowledge bases. In addition, a majority voting technique is used to improve the confidence level in the recognition step. A number of experiments using a prototype implementation and selected gestures resulted in a rate of 100% true recognition. The system also showed to be robust, and flexible as new gesture patterns can be easily included by means of the on-chip training capabilities. An application for this systems is also presented, consisting of four smart cameras used in simultaneous localization and mapping (SLAM) tasks for robotics.

Design of a real-time gesture recognition system: high performance through algorithms and software

The embedded systems group at Princeton University in New Jersey has developed, as an example of smart cameras, a gesture recognition system that can build a complete model of the torso and recognize gestures at 30 frames/s. Designing a real-time gesture recognition system is a complex task that involves many issues such as algorithm design, processing speed, system architecture, and video interface. In this article, the authors describe a method to manage the complexity by decomposing the entire process into different design and implementation phases.

Real-time gesture recognition system and application

In this paper, we consider a vision-based system that can interpret a user's gestures in real time to manipulate windows and objects within a graphical user interface. A hand segmentation procedure first extracts binary hand blob(s) from each frame of the acquired image sequence. Fourier descriptors are used to represent the shape of the hand blobs, and are input to radial-basis function (RBF) network(s) for pose classification. The pose likelihood vector from the RBF network output is used as input to the gesture recognizer, along with motion information. Gesture recognition performances using hidden Markov models (HMM) and recurrent neural networks (RNN) were investigated. Test results showed that the continuous HMM yielded the best performance with gesture recognition rates of 90.2%. Experiments with combining the continuous HMMs and RNNs revealed that a linear combination of the two classifiers improved the classification results to 91.9%. The gesture recognition system was deployed in a prototype user interface application, and users who tested it found the gestures intuitive and the application easy to use. Real time processing rates of up to 22 frames per second were obtained.

A method of model improvement for spotting recognition of gestures using an image sequence

We have developed a real-time gesture recognition system whose models can be taught by only one instruction. Therefore the system can adapt to new gesture performer quickly but it can not raise the recognition rates even if we teach gestures many times. That is because the system could not utilize all the teaching data. In order to cope with the problem, averages of teaching data are calculated. First, the best frame correspondence of the teaching data and the model is obtained by Continuous DP. Next the averages and variations are calculated for each frame of the model. We show the effectiveness of our method in the experiments.