Hand Gesture Detection Research Articles

Gesture-Based Control of Multimedia Player Using Python and OpenCV

The increasing significance of computers in our daily lives, coupled with the rise of ubiquitous computing, has necessitated effective human-computer interaction. Hand gesture recognition systems have emerged as a real-time video-based solution for detecting and interpreting hand gestures, offering intelligent and natural human-computer interaction (HCI) methods. This project focuses on leveraging human hands as input devices for computer operation. Developed using Python and the OpenCV library, the program utilizes a computer webcam to capture and analyze hand shapes and patterns. The program provides real-time feedback by displaying recognized hand gestures on the live video stream. The ultimate outcome of this project is an application that enhances user experiences in contactless systems. The project recognizes and detects human hand motions using the Python computer language through a process flow that includes background subtraction, hand ROI segmentation, contour detection, and finger recognition. Techniques for processing images are used, including hand gesture detection, pattern recognition, thresholding, and contour detection. The processing of incoming photos and the creation of related keystrokes are made possible by OpenCV, a rich set of image processing tools

Empowering human-robot interaction using sEMG sensor: Hybrid deep learning model for accurate hand gesture recognition

In this paper, a novel approach using a Henry Gas Solubility-based Stacked Convolutional Neural Network (HGS-SCNN) for hand gesture recognition using surface electromyography (sEMG) sensors is proposed. The stacked architecture of the CNN model helps to capture both low-level and high-level features, enabling effective representation learning. To begin, we generated a dataset comprising 600 samples of hand gestures. Next, we applied the Discrete Wavelet Transform (DWT) technique to extract features from the filtered sEMG signal. This step allowed us to capture both spatial and frequency information, enhancing the discriminative power of the extracted features. Extensive experiments are conducted to evaluate the performance of the proposed HGS-SCNN model. In addition, the obtained results are compared with state-of-the-art techniques, namely AOA-SCNN, GWO-SCNN, and WOA-SCNN. The comparative analysis demonstrates that the HGS-SCNN outperforms these existing methods, achieving an impressive accuracy of 99.3%. The experimental results validate the effectiveness of our proposed approach in accurately detecting hand gestures. The combination of DWT-based feature extraction and the HGS-SCNN model offers robust and reliable hand gesture recognition, thereby opening new possibilities for intuitive human-machine interaction and applications requiring gesture-based control.

A IMPLEMENTATION ON IMPLEMTATION OF VISION GLIDE TECHNIQUE FOR SMOOTH NAVIGATION WITH CAMERA BASED VIRTUAL MOUSE

As artificial intelligence technology has advanced, it has become commonplace to employ hand gesture detection to control virtual objects. The suggested system in this research is a hand gesture-controlled virtual mouse that uses AI algorithms to recognize hand gestures and transform them into mouse movements. People who have trouble using a conventional mouse or keyboard can use the system to provide an alternate interface. The suggested method takes pictures of the user's hand with a camera, which an AI program then utilizes to identify the motions the user is making. Since the development of computer technology, the method for constructing a process of human- computer interaction is advancing. The mouse is one of the best pieces of HCI (Human-Computer Interaction) technology ever created. This article suggests an HCI-based virtual mouse system that makes use of hand movements and computer vision. webcam or built-in camera recordings of gestures that have been subjected to a color segmentation and detection procedure. This system uses a webcam or an integrated camera to capture frames. It then processes the frames to make them trackable, identifies various user motions, and executes mouse functions. In order to make the interaction more effective and dependable, this study suggests a vision-based system to control various mouse behaviors, such as left and right clicking, using hand gestures. This system uses a webcam or an integrated camera to capture frames. It then processes the frames to make them trackable, identifies various user motions, and executes mouse functions. Therefore, the suggested mouse solution removes the need for a device to use a mouse. Therefore, it can be seen that the development of HCI technology is advantageous. Keywords- HCI (Human Computer Interaction), Hand Gesture, Gesture Recognition, OpenCV, Media-pipe, pyAutoGUI.

Using end-to-end learning and PyAutoGUI to apply gesture recognition for human-computer interaction

Contact-less human-machine interaction is becoming increasingly important due to the growing number of special environmental needs and accessibility situations. Gesture recognition has also been a hot topic in computer vision and machine learning in recent years. In this paper, a real-time computer manipulation system based on hand gesture recognition is studied and deployed. A relatively mature end-to-end target recognition model, the YOLOv5 model, is trained in this paper to achieve real-time detection and recognition of hand gestures. According to the result of the recognition, it is translated into the corresponding operation on the computer according to a set of rules, and then PyAutoGUI is used to actually control the computer. At the end of the research, the trained YOLOv5 model exhibited excellent performance and verified the feasibility and scalability of the solution. This is a good inspiration for developing a more convenient and efficient related software.

Weight thresholded regularized robust coding for vision sensing-based hand gesture detection in collaborative robotics

Vision sensing-based hand gesture recognition is considered an important contemporary research problem of collaborative robotics in a human-robot coexisting environment. The problem becomes more complex in challenging environments, where a robot needs to correctly recognize a human gesture to perform navigation or some other designated job, even in the presence of poor illumination conditions, occlusion, etc. This work proposes a novel approach of successfully utilizing regularized robust coding (RRC)-based models to solve such hand gesture detection problems in real-life, challenging situations. The RRC model is used for robust regression of a signal or image and is known as an improvement over the classical sparse representation-based classification (SRC) model. In this work, we propose three novel variants of weight thresholding mechanism in conjunction with RRC (named WTRRC algorithms) which essentially employ more error-tolerant concepts of logistic functions for weight update, when the dictionary is formed by real-world, photometrically irregular hand gesture images. Extensive case studies in real-world environments (i) with poor illumination, and (ii) with both poor illumination and occlusions firmly establish superior performances of the proposed WTRRC variants compared to other state-of-the-art algorithms in such collaborative robotics problems.

A REVIEW ON IMPLEMTATION OF VISION GLIDE TECHNIQUE FOR SMOOTH NAVIGATION WITH CAMERA BASED VIRTUAL MOUSE

As artificial intelligence technology has advanced, it has become commonplace to employ hand gesture detection to control virtual objects. The suggested system in this research is a hand gesture-controlled virtual mouse that uses AI algorithms to recognize hand gestures and transform them into mouse movements. People who have trouble using a conventional mouse or keyboard can use the system to provide an alternate interface. The suggested method takes pictures of the user's hand with a camera, which an AI program then utilizes to identify the motions the user is making. Since the development of computer technology, the method for constructing a process of human-computer interaction is advancing. The mouse is one of the best pieces of HCI (Human-Computer Interaction) technology ever created. This article suggests an HCI-based virtual mouse system that makes use of hand movements and computer vision. webcam or built-in camera recordings of gestures that have been subjected to a color segmentation and detection procedure. This system uses a webcam or an integrated camera to capture frames. It then processes the frames to make them trackable, identifies various user motions, and executes mouse functions. In order to make the interaction more effective and dependable, this study suggests a vision-based system to control various mouse behaviors, such as left and right clicking, using hand gestures. This system uses a webcam or an integrated camera to capture frames. It then processes the frames to make them trackable, identifies various user motions, and executes mouse functions. Therefore, the suggested mouse solution removes the need for a device to use a mouse. Therefore, it can be seen that the development of HCI technology is advantageous. Keywords- HCI (Human Computer Interaction), Hand Gesture, Gesture Recognition, OpenCV, Media-pipe, pyAutoGUI.

WiFi-Based Lightweight Gesture Recognition for Coal Miners

With the advancement of smart mines, the need for gesture recognition for remote interaction between underground workers and machines has become crucial. However, traditional gesture recognition techniques require complex models that are very difficult to be deployed to the edge. To address this challenge, a gesture recognition method based on knowledge distillation is proposed in this study. First, the CSI ratio model is used to eliminate phase error and environmental noise, followed by the application of discrete wavelet transform to eliminate hardware noise interference. Then, the processed data is adaptively segmented using the principal component analysis and local anomaly factor algorithm to eliminate redundant static components. After that the processed CSI data is transformed into images using the relative position matrix method. Finally, knowledge distillation is employed to migrate knowledge from a teacher model to a student model, reducing the number of model parameters. Experiments conducted on the proposed method showed that it can achieve a recognition accuracy of 94.2% for hand gesture detection, which meets the requirement for gesture recognition in the mining industry.

REAL-TIME HAND GESTURE DETECTION AND RECOGNITION USING CONVOLUTIONAL ARTIFICIAL NEURAL NETWORKS

Real-time hand gesture detection is an interesting topic in pattern recognition and computer vision. In this study, we propose the use of a Convolutional Neural Network (CNN) to detect and recognize hands in real-time. Our goal is to develop a system that can accurately identify and interpret user gestures in real-time. The proposed approach involves two main stages, namely hand gesture recognition and gesture recognition. For stage detection, we use the CNN architecture to recognize hands in the video. We train the CNN model using a dataset containing various hand gestures. Once a hand is detected, we extract the relevant hand region and proceed to the gesture recognition stage. The gesture recognition stage involves training and testing CNN models for different hand signal recognition. We use a hand gesture dataset that contains a variety of common hand signals. The experimental results show that the proposed system can detect and recognize hand movements in real-time with satisfactory accuracy. Although there are still some challenges that need to be overcome, this research provides a solid foundation for further development in real-time hand gesture recognition.

Robust Gesture Recognition and Classification for Visually Impaired Persons Using Growth Optimizer with Deep Stacked Autoencoder

Visual impairment affects the major population of the world, and impaired vision people need assistance for their day-to-day activities. With the enormous growth and usage of new technologies, various devices were developed to help them with object identification in addition to navigation in the indoor and outdoor surroundings. Gesture detection and classification for blind people aims to develop technologies to assist those people to navigate their surroundings more easily. To achieve this goal, using machine learning and computer vision techniques is a better solution to classify and detect hand gestures. Such methods are utilized for finding the shape, position, and movement of the hands in real-time. With this motivation, this article presents a robust gesture recognition and classification using growth optimizer with deep stacked autoencoder (RGRC-GODSAE) model for visually impaired persons. The goal of the RGRC-GODSAE technique lies in the accurate recognition and classification of gestures to assist visually impaired persons. The RGRC-GODSAE technique follows the Gabor filter approach at the initial stage to remove noise. In addition, the RGRC-GODSAE technique uses the ShuffleNet model as a feature extractor and the GO algorithm as a hyperparameter optimizer. Finally, the deep stacked autoencoder model is exploited for the automated recognition and classification of gestures. The experimental validation of the RGRC-GODSAE technique is carried out on the benchmark dataset. The extensive comparison study showed better gesture recognition performance of the RGRC-GODSAE technique over other deep learning models.

Virtual Mouse and Keyboard for Computer Interaction by Hand Gestures Using Machine Learning

Human-computer interaction has changed since the advent of computer technology. Gestures are a useful way to communicate, and the Covid-19 era had an impact on us. Both the keyboard and the mouse are tools used to communicate with computers. Here, we’ve attempted to use hand gestures to interact with the mouse and keyboard. Eventually, get rid of the electronics. Consequently, use a virtual keyboard and your finger to move the mouse cursor. Using different hand gestures, actions like clicking, dragging, and typing data will be carried out. A webcam is the IOT device required to accomplish this. The output from the camera will be displayed on the system’s screen so that the user can fine-tune it. We employ tools like Python, Media- Pipe, and Open-CV. The Media-Pipe library offers features that improve the model’s effectiveness and is particularly helpful in AI projects. The user will be able to move the computer cursor with various hand motions, type on the virtual keyboard while holding coloured caps or tapes, and left-click and drag objects. In this research, we suggest a hand gesture detection system for a natural human computer interface that can control a virtual mouse and keyboard.

Computer vision-based hand gesture recognition for human-robot interaction: a review

As robots have become more pervasive in our daily life, natural human-robot interaction (HRI) has had a positive impact on the development of robotics. Thus, there has been growing interest in the development of vision-based hand gesture recognition for HRI to bridge human-robot barriers. The aim is for interaction with robots to be as natural as that between individuals. Accordingly, incorporating hand gestures in HRI is a significant research area. Hand gestures can provide natural, intuitive, and creative methods for communicating with robots. This paper provides an analysis of hand gesture recognition using both monocular cameras and RGB-D cameras for this purpose. Specifically, the main process of visual gesture recognition includes data acquisition, hand gesture detection and segmentation, feature extraction and gesture classification, which are discussed in this paper. Experimental evaluations are also reviewed. Furthermore, algorithms of hand gesture recognition for human-robot interaction are examined in this study. In addition, the advances required for improvement in the present hand gesture recognition systems, which can be applied for effective and efficient human-robot interaction, are discussed.

Real-time continuous detection and recognition of dynamic hand gestures in untrimmed sequences based on end-to-end architecture with 3D DenseNet and LSTM

Real-time continuous detection and recognition of dynamic hand gestures in untrimmed sequences based on end-to-end architecture with 3D DenseNet and LSTM

Static Hand Gesture Recognition Using Novel Convolutional Neural Network and Support Vector Machine

Hand tracking and identification through visual means pose a challenging problem. To simplify the identification of hand gestures, some systems have incorporated position markers or colored bands, which are not ideal for controlling robots due to their inconvenience. The motion recognition problem can be solved by combining object identification, recognition, and tracking using image processing techniques. A wide variety of target detection and recognition image processing methods are available. This paper proposes novel CNN-based methods to create a user-free hand gesture detection system. The use of synthetic techniques is recommended to improve recognition accuracy. The proposed method offers several advantages over existing methods, including higher accuracy and real-time hand gesture recognition suitable for sign language recognition and human-computer interaction. The CNN automatically extracts high-level characteristics from the source picture, and the SVM is used to classify these features. This study employed a CNN to automatically extract traits from raw EMG images, which is different from conventional feature extractors. The SVM classifier then determines which hand gestures are being made. Our tests demonstrate that the proposed strategy achieves superior accuracy compared to using only CNN.

Communication for Blind Individual Using CNN and Text Summarization

Abstract: In order to bridge the communication gap between the deaf and the dumb, sign language is a special kind of communication language. There are several signs in each sign language with differences in palm size, shape, motion, and positioning of the hand, all of which are important components of each sign. Many different researchers have proposed numerous applications. Utilising deep learning principles, several notable advancements have been made in these applications during the last few years. Throughout this survey, we evaluated various hand gesture detection applications utilising recent deep learning theories. Despite significant advancements in hand gesture detection accuracy, there are still a number of issues that need to be handled. In our proposal,

Controlled Hand Gestures using Python and OpenCV

Abstract: Due to its communal nature, gesture recognition has been used in recent trends to develop machines. Gestures are a type of verbal communication that allows humans and computers to communicate with one another. Artificial intelligence makes heavy use of hand gesture detection to improve functionality and person commerce. Then, to carry out particular tasks, we used counterplotted action dyads and some Python libraries (OpenCV, cvzone) that aid in image capture, pre-processing, and discovery

Human Computer Interaction with detection of Hand Gesture to improve Artistic creativity

This research purpose is to improve the interest of artistic creativity for school or college’s students. Find new ways to use virtual reality technology to improve art professional education. Drawing using hand gestures is where we can draw by just capturing the motion of a colored marker with a camera. Discuss how computer vision can play an important role in art education. Using this we have created a simulation for art students or users which helps them to increase their interest to use modern technology which is going to help them to improve their artistic creativity. The Virtual Paint is where students or users can draw by just capturing the motion of a colored marker using a Camera.

An Approach to Convert Sign to Text for Impaired People

Human Computer Interface is the study of how humans and computers interact. Hand gestures are a great way to communicate with people when they don’t understand exactly what we are saying. Understanding hand gestures is essential to make sure the listener understands what we are saying. The main idea of our project is to try different approaches to hand gesture recognition. This proposed work first with radar data and then with camera sensor to achieve hand gesture recognition. First, we tried to build hand gesture recognition using radar data, and since most people don’t know sign language and few interpreters, we developed an approach to real-time approach for American Sign Language based on neural networks finger spelling followed by another model with Media Pipe. We propose a complex neural network method to detect hand gestures of human behaviour from camera recorded images. The hand gesture first goes through the filter and after applying the filter the gesture goes through a classifier that predicts which type of hand gesture it is. In an existing system radar unable to detect static gestures in our approach, a deep learning-based image captioning algorithm captures both static and dynamic gestures through Media Pipe.

Using a Gesture Recognition Modeling approach to improve graphical user interface

The gesture recognition technology based on visual detection employs image processing for detection, segmentation, tracking, and recognition of hand gestures. Static and dynamic gesture recognition are the two types of gesture recognition. In this paper, we present our hand gesture interface consisting of three main layers: Hand Gesture Interface, Mapping Gesture, Action, and Graphical User Interface with our approach named IGMA (Improving Graphical User Interface Based on Gesture Recognition Modeling Approach). IGMA organized in two stages: the modeling stage and the code generation stage. The model is Domain Specific Model; it allows action and gesture modeling. At the code generation stage, a specific code generator transforms all models to final source code using a framework specifically designed for a given target platform. Finally, to simplify the implementation of this method, we propose defining a process for improving user interface control based on gesture recognition. We validate our approach by performing experiments on a gastronomy application. The result is an adaptation framework that can guide software engineers in developing Graphical User Interface Based on Gesture Recognition.

Hand Gesture Recognition System for Deaf and Dumb People

Deaf and dumb people communicate using language known as Sign language. There are various sign languages but comparing to other sign languages, Indian Sign Language interpretation has got least mindfulness by researchers. Alphabets in English sign language and various phrases can be recognized using the application. It deals with images of bare hands. Hand-Gesture recognition is the ability to recognize hand-gesture. The hand gesture recognition system is solution to this problem which uses the image of hand gesture and recognize the sign present in the image. These features based on gesture analysis of the hand on which image of sign is shown and it extract slant and slope information. System provides an opportunity for deaf and dumb people to communicate with normal people effortlessly. Our research focuses on recognition of English sign language for vast reach of audience. There have been several upgrades in technology and a lot of research has been done to aid the deaf and dumb people. Aiding the cause, Deep learning along with computer vision is being used to make an impact on this cause. It can help the deaf and dumb people in communicating with others as many of us don’t know what sign language is. In this sign language recognition project, we create an kotlin application, which detects numbers from 1 to 10 and also the English alphabets. The initial step of this system is to build a database of English Sign Language. Hand gesture detection is the most determining step in each hand gesture recognition system since if we get better segmented output, better recognition rates can be achieved. The proposed system also includes efficient and robust hand segmentation and tracking algorithm to achieve finer recognition rates. Various isolated words from the Standard English sign language have been recognized using a large set of samples. In proposed system, we intend to recognize some very basic elements of sign language and to translate them to text .

A Machine Learning Based Full Duplex System Supporting Multiple Sign Languages for the Deaf and Mute

This manuscript presents a full duplex communication system for the Deaf and Mute (D-M) based on Machine Learning (ML). These individuals, who generally communicate through sign language, are an integral part of our society, and their contribution is vital. They face communication difficulties mainly because others, who generally do not know sign language, are unable to communicate with them. The work presents a solution to this problem through a system enabling the non-deaf and mute (ND-M) to communicate with the D-M individuals without the need to learn sign language. The system is low-cost, reliable, easy to use, and based on a commercial-off-the-shelf (COTS) Leap Motion Device (LMD). The hand gesture data of D-M individuals is acquired using an LMD device and processed using a Convolutional Neural Network (CNN) algorithm. A supervised ML algorithm completes the processing and converts the hand gesture data into speech. A new dataset for the ML-based algorithm is created and presented in this manuscript. This dataset includes three sign language datasets, i.e., American Sign Language (ASL), Pakistani Sign Language (PSL), and Spanish Sign Language (SSL). The proposed system automatically detects the sign language and converts it into an audio message for the ND-M. Similarities between the three sign languages are also explored, and further research can be carried out in order to help create more datasets, which can be a combination of multiple sign languages. The ND-M can communicate by recording their speech, which is then converted into text and hand gesture images. The system can be upgraded in the future to support more sign language datasets. The system also provides a training mode that can help D-M individuals improve their hand gestures and also understand how accurately the system is detecting these gestures. The proposed system has been validated through a series of experiments resulting in hand gesture detection accuracy exceeding 95%.