Hand Gesture Detection Research Articles

Hand Gesture Enabled Operation for Computer Using Deep Learning

This project aims to develop a system that detects hand gestures and prints corresponding instructions for computer operators. The system utilizes Deep learning algorithms CNN and mediapipe to recognize hand gestures, and then prints instructions for the operator to perform specific computer operations.Operations like Notepad Open,Shutdown,Volume Up,Volume Down etc.The system will consist of a webcam, a Deep learning model, and a printer. The webcam will capture images of hand gestures, which will be processed using the Deep learning model to identify the specific gesture. The system will then print the corresponding instruction on the printer. The system will be designed to be user-friendly and adaptable to different environments and hardware configurations.The system aims to improve efficiency, reduce errors, and enhance accessibility for computer operators. The proposed system consists of a hand gesture detection module, instruction printing module, and computer operation module. The system is trained using a dataset of hand gestures and corresponding computer operations. Experimental results show that the system achieves high accuracy in gesture recognition and instruction printing. The system has potential applications in various fields, including gaming, education, and healthcare. Key Words: Hand gesture detection, Deep learning, instruction printing, computer operation, accessibility, Mediapipe, CNN.

STATIC SIGN LANGUAGE TRANSLATOR USING HAND GESTURE AND SPEECH RECOGNITION

Communication between ordinary and deaf people often has issues because ordinary people lack knowledge of sign language. This research aims to help ordinary people communicate with hearing impaired (deaf) people using sign language. This research aims to produce an Android-based mobile app that can translate static sign language using hand movements into text and also convert the spoken voice into sign language using speech recognition. The framework in this research for hand gesture detection uses the MediaPipe software program. This framework allows the creation of apps that translate hand movements into text that help ordinary people understand sign language. Speech recognition in this research uses the Android speech library. This research succeeded in detecting static letters of the alphabet from the a-z and numbers 0-9. Tests of 540 hand gestures carried out in the morning, afternoon, evening, and night had an average detection time of 4.37 seconds. The fastest object detection times were in the morning at a distance of 30 cm with an average detection time of 2.5 seconds. Based on acceptance testing, 83.13% of the features in this static sign language translator have met the users ’users’ needs when communicating with the deaf using sign language.

Real-Time Hand Gesture Monitoring Model Based on MediaPipe's Registerable System.

Hand gesture recognition plays a significant role in human-to-human and human-to-machine interactions. Currently, most hand gesture detection methods rely on fixed hand gesture recognition. However, with the diversity and variability of hand gestures in daily life, this paper proposes a registerable hand gesture recognition approach based on Triple Loss. By learning the differences between different hand gestures, it can cluster them and identify newly added gestures. This paper constructs a registerable gesture dataset (RGDS) for training registerable hand gesture recognition models. Additionally, it proposes a normalization method for transforming hand gesture data and a FingerComb block for combining and extracting hand gesture data to enhance features and accelerate model convergence. It also improves ResNet and introduces FingerNet for registerable single-hand gesture recognition. The proposed model performs well on the RGDS dataset. The system is registerable, allowing users to flexibly register their own hand gestures for personalized gesture recognition.

Retraction Note: An efficient method for human hand gesture detection and recognition using deep learning convolutional neural networks

Retraction Note: An efficient method for human hand gesture detection and recognition using deep learning convolutional neural networks

Hand Gestures Recognition Model Using Adaptive Feature Extraction with Attention-Based Hybrid Deep Learning via Optimization Strategy

Research on continuous hand gesture recognition is motivated by applications of deaf people to provide effective communication between one another. Numerous techniques are adapted for effectively identifying the hand gesture. Moreover, the collection of data becomes a challenging task in the existing models. In some instances, hand gesture in low-resolution images becomes a complicated task. In this research work, a deep learning-based hand gesture detection model is designed using hand data points to uncover patterns that are used to convey information. Initially, hand gesture images are obtained from public sources. The collected inputs are preprocessed through Contrast-Limited Adaptive Histogram Equalization and filtering to eliminate the unnecessary blurs in the images. The preprocessed images undergo hand segmentation using four methods color space transformation, skin color detection, active contour, and morphological operation. The segmented image is subjected to data point feature extraction, where the Adaptive Weighted Scale-Invariant Feature Transform is utilized for further enhancement. The hand data point-extracted features are given to the recognition of hand gestures with the support of an attention-based hybrid network, where the Attention-based Hybrid 1D Convolutional Neural Network with Recurrent Neural Network recognizes the hand gestures from the hand data points. In the hybridization network, the values are optimized with the help of the investigated Adaptive Dandelion White Shark Optimizer to enhance recognition effectiveness. The test results are validated with the existing hand gesture recognition models using diverse evaluation metrics. The findings of the recommended method show 97% and 99% in terms of accuracy and NPV. Thus, the developed model is validated to outperform significant performance rather than the existing models.

Enhancing unmanned ground vehicle performance in SAR operations: integrated gesture-control and deep learning framework for optimised victim detection.

In this study, we address the critical need for enhanced situational awareness and victim detection capabilities in Search and Rescue (SAR) operations amidst disasters. Traditional unmanned ground vehicles (UGVs) often struggle in such chaotic environments due to their limited manoeuvrability and the challenge of distinguishing victims from debris. Recognising these gaps, our research introduces a novel technological framework that integrates advanced gesture-recognition with cutting-edge deep learning for camera-based victim identification, specifically designed to empower UGVs in disaster scenarios. At the core of our methodology is the development and implementation of the Meerkat Optimization Algorithm-Stacked Convolutional Neural Network-Bi-Long Short Term Memory-Gated Recurrent Unit (MOA-SConv-Bi-LSTM-GRU) model, which sets a new benchmark for hand gesture detection with its remarkable performance metrics: accuracy, precision, recall, and F1-score all approximately 0.9866. This model enables intuitive, real-time control of UGVs through hand gestures, allowing for precise navigation in confined and obstacle-ridden spaces, which is vital for effective SAR operations. Furthermore, we leverage the capabilities of the latest YOLOv8 deep learning model, trained on specialised datasets to accurately detect human victims under a wide range of challenging conditions, such as varying occlusions, lighting, and perspectives. Our comprehensive testing in simulated emergency scenarios validates the effectiveness of our integrated approach. The system demonstrated exceptional proficiency in navigating through obstructions and rapidly locating victims, even in environments with visual impairments like smoke, clutter, and poor lighting. Our study not only highlights the critical gaps in current SAR response capabilities but also offers a pioneering solution through a synergistic blend of gesture-based control, deep learning, and purpose-built robotics. The key findings underscore the potential of our integrated technological framework to significantly enhance UGV performance in disaster scenarios, thereby optimising life-saving outcomes when time is of the essence. This research paves the way for future advancements in SAR technology, with the promise of more efficient and reliable rescue operations in the face of disaster.

Static and Dynamic Hand Gestures: A Review of Techniques of Virtual Reality Manipulation.

This review explores the historical and current significance of gestures as a universal form of communication with a focus on hand gestures in virtual reality applications. It highlights the evolution of gesture detection systems from the 1990s, which used computer algorithms to find patterns in static images, to the present day where advances in sensor technology, artificial intelligence, and computing power have enabled real-time gesture recognition. The paper emphasizes the role of hand gestures in virtual reality (VR), a field that creates immersive digital experiences through the Ma blending of 3D modeling, sound effects, and sensing technology. This review presents state-of-the-art hardware and software techniques used in hand gesture detection, primarily for VR applications. It discusses the challenges in hand gesture detection, classifies gestures as static and dynamic, and grades their detection difficulty. This paper also reviews the haptic devices used in VR and their advantages and challenges. It provides an overview of the process used in hand gesture acquisition, from inputs and pre-processing to pose detection, for both static and dynamic gestures.

IoT-based Smart Screen Control Using Hand Gesture

In today's dynamic educational environment, the demand for innovative tools to enhance teaching methodologies is paramount. This research paper proposes a novel IoT-based smart screen control system tailored explicitly for educational settings [10]. At its core lies hand gesture recognition technology, powered by the MediaPipe library, facilitating seamless real-time detection and tracking of hand gestures for intuitive presentation control. Integrating camera sensors and microcontrollers [14], the system ensures precise data transmission to computers interfacing with smart boards. An advanced algorithm synchronizes hand feed and movement flawlessly [14], offering users a glitch-free experience. The user-friendly interface empowers educators to customize settings and gestures, fostering personalized interactions. Furthermore, the system enables control over various types of presentations and slides, as well as zooming functionality within any opened interface [4]. Through a comprehensive literature review and implementation of state-of-the-art methodologies [12], this research revolutionizes human-computer interaction in educational environments. By adhering to stringent citation guidelines and considering the quality and relevance of references, this paper contributes to advancing knowledge in the field [1, 2, 4, 10, 12, 14]. The proposed system sets a new standard for efficiency and effectiveness in educational settings, promising to reshape the teaching landscape and inspire creativity in classrooms. Key Words: Media Pipe, IoT, camera sensors, microcontrollers, presentation control, customization, educational environments, human-computer interaction, OpenCV, Python.

Multifunctional Nanocomposite Yield-Stress Fluids for Printable and Stretchable Electronics.

Compliant materials are crucial for stretchable electronics. Stretchable solids and gels have limitations in deformability and durability, whereas active liquids struggle to create complex devices. This study presents multifunctional yield-stress fluids as printable ink materials to construct stretchable electronic devices. Ionic nanocomposites comprise silica nanoparticles and ion liquids, while electrical nanocomposites use the natural oxidation of liquid metals to produce gallium oxide nanoflake additives. These nanocomposite inks can be printed on an elastomer substrate and stay in a solid state for easy encapsulation. However, their transition into a liquid state during stretching allows ultrahigh deformability up to the fracture strain of the elastomer. The ionic inks produce strain sensors with high stretchability and temperature sensors with high sensitivity of 7% °C-1. Smart gloves are further created by integrating these sensors with printed electrical interconnects, demonstrating bimodal detection of temperatures and hand gestures. The nanocomposite yield-stress fluids combine the desirable qualities of solids and liquids for stretchable devices and systems.

Controlling 3D Models in VR using Hand Gestures

Abstract: The fusion of hand gesture detection technology with virtual reality (VR) environments enables users to interact organically with three-dimensional models. Utilizing VR headsets equipped with advanced hand tracking algorithms, individuals can engage with virtual objects through intuitive gestures, enriching their immersive experiences. Software development entails carrying out sophisticated gesture recognition algorithms and the assignment of specific actions to these gestures, such as grasping, rotating, and resizing 3D models. These interactions mirror real-world manipulations, empowering users to seamlessly explore and modify digital content. Furthermore, gestural menu systems streamline user interactions by offering convenient access to additional features. Rigorous testing and optimization processes ensure precision, responsiveness, and optimal performance, enhancing the overall user satisfaction.

Hand Motion: Mapping Gestures Through Recognition

Abstract: In our everyday dealings with friends, family and co-workers, gestures are crucial as they offer an easy and innate way of communicating. From simple hand movements to greet someone or acknowledge friends from a distance, gestures form part of our daily engagements. Everybody, recognizes gestures as the extension of our body language. If this concept is applied to human-computer interaction, it becomes possible to have hand gesture detection through vision-based technology. In this case, certain movements are performed before a camera with each movement corresponding to specific actions.

Hand Gesture Gaming

Hand gesture recognition has emerged as a promising interface for interactive gaming applications, offering intuitive and immersive experiences. This research paper explores the development and implementation of a hand gesture recognition system using Python for gaming purposes. The study investigates various machine learning techniques, particularly convolutional neural networks (CNNs), for accurate detection and classification of hand gestures from video input. The research methodology involves collecting a diverse dataset of hand gesture samples, encompassing common gaming gestures such as swipes, taps, and rotations. Preprocessing techniques are employed to enhance the quality and consistency of the dataset. Subsequently, deep learning architectures, including CNNs, are trained on the dataset to recognize and classify hand gestures in real-time.

RADFormers for Detection and Classification of Hand Gestures

RADFormers for Detection and Classification of Hand Gestures

A Hand Gesture-Operated System for Rehabilitation Using an End-to-End Detection Framework

We propose a hand gesture-operated system as an AI application to relieve discomfort and restore function in hand and arm movements caused by injuries and nerve and muscle complications. The system trains patients with hand exercises, such as performing hand gestures accurately, traversing within specified bounds, and operating a hand gesture calculator. However, the system requires accurate hand gesture detection, which is impeded by background clutter and variations in illumination and in the hand's size and angle. To address this, we developed a robust hand detection module that uses a single-stage transformer deep network. The transformer network encodes global information and uses bipartite matching to reduce the frequency of spurious detections. It drives a regression head and a classification head to localize the hand gesture in a bounding box and assign it a class label. Hand keypoints are also detected to support drawing, path traversal, and calculator use. The approach is evaluated on two benchmark datasets: OUHANDS and NUS. The method yields 89.6% accuracy for OUHANDS and 100% for NUS. These results indicate that precise hand detection can support a robust system for rehabilitation through hand exercises. Our experiments confirm that the users' hand function progressively improved.

Breaking Barriers: Hand Gesture Vocalizer for the Deaf and Mute

This study addresses communication challenges faced by deaf and mute individuals by exploring the feasibility of utilizing flex sensors and the Random Forest algorithm for hand gesture vocalization. The background underscores the significance of accessible communication tools in enhancing the lives of those with hearing and speech impairments. The study's purpose is to assess the effectiveness of flex sensors in detecting hand gestures and the Random Forest algorithm's potential to generate vocalized speech corresponding to these gestures. The methodology involves data collection from flex sensors through Arduino, Random Forest model training, and accuracy evaluation in gesture recognition. Promising results indicate the model's high accuracy in classifying diverse hand gestures. The study emphasizes the technology-driven solution's importance in bridging communication gaps for those with impairments. Combining flex sensors and the Random Forest algorithm offers an intuitive communication tool, transforming interactions for deaf and mute individuals. Consideration for real-world scenarios and user diversity during system development is highlighted, crucial for practical accuracy. Beyond individual communication, the study's implications span education, employment, and social integration for people with disabilities. Implementing this technology in education fosters inclusive environments, empowering deaf and mute students to engage actively. The integration of flex sensors and the Random Forest algorithm holds immense promise, revolutionizing communication, and life quality. As an accessible gesture-based vocalization tool, it can reshape societal perspectives, fostering inclusivity and empathy. The study advocates continuous research and development, urging widespread technology adoption to create an inclusive society valuing diversity.

Three Sliding Probes Placed on Forelimb Skin for Proprioceptive Feedback Differentially yet Complementarily Contribute to Hand Gesture Detection and Object-Size Discrimination

The purpose was to assess the effectiveness of three sliding tactile probes placed on the forelimb skin to provide proprioceptive feedback for the detection of hand gestures and discrimination of object size. Tactile contactors representing the first three fingers were driven along the proximodistal axis by linear servo motors. Twenty healthy subjects were involved in the gesture detection test, with 10 of them also participating in the object-size discrimination task. Motors were controlled by computer in the first four sessions of the gesture detection experiment, while the fifth session utilized a sensorized glove. Both the volar and dorsal sides of the forearm were examined. In the object-size discrimination experiment, the method was exclusively assessed on the volar surface under four distinct feedback conditions, including all fingers and each finger separately. The psychophysical data were further analyzed using a structural equation model (SEM) to evaluate the specific contributions of each individual contactor. Subjects consistently outperformed the chance level in detecting gestures. Performance improved up to the third session, with better results obtained on the volar side. The performances were similar in the fourth and fifth sessions. The just noticeable difference for achieving a 75% discrimination accuracy was found to be 2.90 mm of movement on the skin. SEM analysis indicated that the contactor for the index finger had the lowest importance in gesture detection, while it played a more significant role in object-size discrimination. However, all fingers were found to be significant predictors of subjects' responses in both experiments, except for the thumb, which was deemed insignificant in object-size discrimination. The study highlights the importance of considering the partial contribution of each degree of freedom in a sensory feedback system, especially concerning the task, when designing such systems.

Gesture Controlled Virtual Mouse with Voice Assistant

The usage of hand gestures to operate digital devices has grown in recent years, and developing accurate gesture recognition systems using computer vision technology is becoming easier. Voice assistants have also become more common, with developments in artificial intelligence (AI) and natural language processing (NLP) allowing for more accurate voice recognition. Our goal is to create a system which combines both gesture-controlled mouse functionality and voice assistant to execute things that a regular mouse and keyboard can. The GCVA (Gesture Controlled Voice Assistant) comes with a gesture-controlled mouse and a voice assistant. A gesture-controlled mouse uses OpenCV and Media-Pipe, whereas a voice assistant uses AI and NLP concepts to recognize the voice commands given by the user. The system should be used by performing hand gestures and executing the right click, left click, drag, drop, volume control and computer cursor functions, thereby omitting the need for a physical mouse. The success of this system is observed in terms of practicality, stability, and compatibility with physical mechanisms. The voice assistant works and responds to pre-defined commands present in the system. The GCVA system was tested by four different users in order to generate the accuracy scores of mouse gestures and response time graphs for voice assistant. The system yields an accuracy rate of 98%. The system's accuracy and usability showed that hand gesture detection and voice assistants have the potential to replace traditional input devices in the future.

Design and Implementation of Hand Gesture Detection System Using HM Model for Sign Language Recognition Development

Design and Implementation of Hand Gesture Detection System Using HM Model for Sign Language Recognition Development

Real-Time Sign Language Fingerspelling Recognition using Convolutional Neural Network

: Sign language allows mute people to communicate, problem occurs when a conversationalist fails to understand it. Despite efforts to address this problem, an effective solution is not yet found. In this work, Convolutional Neural Network (CNN) was trained on two different datasets separately- binary and Red Blue Green )RGB(, each contains 25,900 images of Nigerian Sign Language. A deep neural pre-trained module was used to detect hand gestures in the video feed which tackled the issue of complex backgrounds, also showed excellent detection in dimly lit areas. The accuracies of (98.95%, 76%) and (98.87%, 98.85%) were obtained respectively on the training and the validation sets. The real time system developed implemented both models as a single system which makes it a unique one.

Real-Time Sign Language Fingerspelling Recognition using Convolutional Neural Network

Sign Language allows mute people to communicate, problem occurs when a conversationalist fails to understand it. Despite efforts to address this problem, an effective solution is not yet found. In this work, Convolutional Neural Network was trained on two different datasets separately- Binary and Red Blue Green )RGB(, each contains 25,900 images of Nigerian Sign Language. A Deep Neural pre-trained module was used to detect hand gestures in the video feed which tackled the issue of complex backgrounds, also showed excellent detection in dimly lit areas. The accuracies of (98.95%, 76%) and (98.87%, 98.85%) were obtained respectively on the training and the validation sets. The real time system developed implemented both models as a single system which makes it a unique one.