Gesture Recognition Technology Research Articles

IoT-based Smart Screen Control Using Hand Gesture

In today's dynamic educational environment, the demand for innovative tools to enhance teaching methodologies is paramount. This research paper proposes a novel IoT-based smart screen control system tailored explicitly for educational settings [10]. At its core lies hand gesture recognition technology, powered by the MediaPipe library, facilitating seamless real-time detection and tracking of hand gestures for intuitive presentation control. Integrating camera sensors and microcontrollers [14], the system ensures precise data transmission to computers interfacing with smart boards. An advanced algorithm synchronizes hand feed and movement flawlessly [14], offering users a glitch-free experience. The user-friendly interface empowers educators to customize settings and gestures, fostering personalized interactions. Furthermore, the system enables control over various types of presentations and slides, as well as zooming functionality within any opened interface [4]. Through a comprehensive literature review and implementation of state-of-the-art methodologies [12], this research revolutionizes human-computer interaction in educational environments. By adhering to stringent citation guidelines and considering the quality and relevance of references, this paper contributes to advancing knowledge in the field [1, 2, 4, 10, 12, 14]. The proposed system sets a new standard for efficiency and effectiveness in educational settings, promising to reshape the teaching landscape and inspire creativity in classrooms. Key Words: Media Pipe, IoT, camera sensors, microcontrollers, presentation control, customization, educational environments, human-computer interaction, OpenCV, Python.

Driving Aid for Rotator Cuff Injured Patients using Hand Gesture Recognition

Gesture recognition is a way for computers to understand how humans move and express themselves without using traditional methods like typing or clicking. Instead of relying on text or graphics, gesture recognition focuses on reading body movements, such as those made by the hands or face. Currently, there is a specific interest in recognizing hand gestures by analyzing the veins on the back of the hand. Scientists have found that each person has a unique arrangement of veins beneath the skin of their hand. When the hand moves, the position of these veins changes, and this change is considered a gesture. These gestures are then translated into specific actions or tasks by coding the hand movements. This technology is particularly helpful for individuals with rotator cuff injuries. The rotator cuff is a group of muscles and tendons in the shoulder that can get injured, causing pain and limiting movement. People with these injuries may have difficulty steering a car, especially if their job or sport involves repetitive overhead motions. With gesture recognition technology, a person can control the car by simply moving their wrist, eliminating the need to use the shoulder. In summary, gesture recognition technology reads the unique patterns of hand veins to interpret hand movements, making it a practical solution for individuals with rotator cuff injuries who may struggle with certain tasks, like steering a car.

Computer Vision Gestures Recognition System Using Centralized Cloud Server

Gesture recognition is a progressive technology that enables communication between humans and computers without any physical interaction. In recent years, gesture recognition technology has provided remarkable advancements in the technology world. A computer vision cloud-based gesture recognition system represents the potential of cloud computing to process and analyze the collected data. This paper examines the various growing aspects of gesture recognition using computer vision and cloud technology. Here we introduce a computer vision-based gesture processing unit that works on a cloud server and is used to enable real-time interaction between the physical world and the virtual server. It will help to use high-end gesture processing technology for every kind of device user. All the data will be collected using vision- based devices and monitored and processed on the centralized server. So the user doesn't have to be a highly efficient device. While using this system, the user can experience both the recent cutting-edge cloud infrastructure as well as the recent gesture recognition system. Not only humans but also object gestures can be recognized by using this proposed system. So this paper aims to represent a significant step toward a cloud-based interaction between human gestures and digital devices.

A Hybrid Image Augmentation Technique for User- and Environment-Independent Hand Gesture Recognition Based on Deep Learning

This research stems from the increasing use of hand gestures in various applications, such as sign language recognition to electronic device control. The focus is the importance of accuracy and robustness in recognizing hand gestures to avoid misinterpretation and instruction errors. However, many experiments on hand gesture recognition are conducted in limited laboratory environments, which do not fully reflect the everyday use of hand gestures. Therefore, the importance of an ideal background in hand gesture recognition, involving only the signer without any distracting background, is highlighted. In the real world, the use of hand gestures involves various unique environmental conditions, including differences in background colors, varying lighting conditions, and different hand gesture positions. However, the datasets available to train hand gesture recognition models often lack sufficient variability, thereby hindering the development of accurate and adaptable systems. This research aims to develop a robust hand gesture recognition model capable of operating effectively in diverse real-world environments. By leveraging deep learning-based image augmentation techniques, the study seeks to enhance the accuracy of hand gesture recognition by simulating various environmental conditions. Through data duplication and augmentation methods, including background, geometric, and lighting adjustments, the diversity of the primary dataset is expanded to improve the effectiveness of model training. It is important to note that the utilization of the green screen technique, combined with geometric and lighting augmentation, significantly contributes to the model’s ability to recognize hand gestures accurately. The research results show a significant improvement in accuracy, especially with implementing the proposed green screen technique, underscoring its effectiveness in adapting to various environmental contexts. Additionally, the study emphasizes the importance of adjusting augmentation techniques to the dataset’s characteristics for optimal performance. These findings provide valuable insights into the practical application of hand gesture recognition technology and pave the way for further research in tailoring techniques to datasets with varying complexities and environmental variations.

Research on Dynamic Gesture Recognition and Control System based on Machine Vision

Hand gesture recognition and control is a new type of human-computer interaction that can provide a more convenient and efficient operation mode by utilizing non-contact gesture recognition technology. This paper presents a lightweight dynamic gesture recognition method for intelligent office presentation control. First, we introduce the concept of hand gesture recognition and go over key gesture recognition technologies like classification. The structure, process, and evaluation index of the gesture recognition algorithm are described in detail using a convolutional neural network model. During the experiment's algorithm verification phase, we test and analyze the algorithm using the Python language, compilation environment, and data set. In the control experiment, we evaluated the system's ability to control the office application's start, play, next, previous, and exit functions. We achieve 96.3% accuracy on the test set. Experimental results show that the system can recognize a wide range of hand gestures and accurately control the presentation.

AirMouse and AirWrite Intuitive Digital Interaction

The primary purpose of this project is to redefine the way people will interact with their systems more easily, and this will be achieved by eliminating the touchpads and keyboards to connect to the computer. This cutting-edge system eliminates the need for a regular mouse or a keyboard, allowing for easier device user-interface adaptation. The implementation of the latest gesture recognition technology lets the system not only pick up and decipher the movements of users' hands but also translate them into wider operations that are similar to those of a mouse and keyboard. Central to the design of the metaphoric system is the "virtual mouse," which enables the cursor movement and the performance of various mouse buttons without touching anything physically. So, it is a virtual mouse with an on-the-screen cursor that reacts to hand gestures in real-time. The system utilizes certain hand gestures enough to enable the effortless execution of actions commonly associated with a traditional mouse including left-clicking, right-clicking, and dragging across the screen. The virtual mouse feature is enhanced by the air writing technique, commonly acknowledged as a milestone in input development. This capability precisely reads and decodes the movements developed in the space by the fingertips while conveying them to the screen as written words. The air writing system would predict the characters from the user's gestures, creating a fresh, secure, and user-friendly method that favors the option of air typing rather than relying on a physical keyboard. This project brings innovation in human-computer interaction when implemented using powerful computer vision and machine learning techniques and promotes accessibility and usability of the system where the traditional inputs become inapplicable. On the other hand, its feasible justifications range from medical uses to pioneering in virtual reality, gaming, and education that drives brilliantly the new options in interaction across various fields.

Hand Gesture Controller

Abstract: The rapid evolution of human-computer interaction has spurred significant progress in gesture recognition technologies, placing a specific emphasis on diverse applications. This paper highlights key advancements in machine learning algorithms tailored for gesture recognition, including deep learning approaches that have notably improved the accuracy and robustness of hand tracking systems. Furthermore, the integration of hand gesture control into wearable devices and its implications for everyday technology usage are thoroughly examined. This paper relies on the formidable capabilities of Deep Learning, specifically Convolutional Neural Networks and Recurrent Neural Networks, to decipher and translate hand gestures into actionable directives. The implementation of the Hand Gesture Controller necessitates the integration of a camera or sensor module capable of capturing intricate hand movements. Real-time image processing and feature extraction are essential components, facilitating the provision of input data to the Deep Learning model. As technology progresses, this interface emerges as a versatile tool capable of enhancing productivity and inclusivity across various domains. The paper concludes with a discussion on future directions in hand gesture controller development, exploring anticipated technological advancements, novel use cases, and the potential for increased accessibility. In summary, the "Hand Gesture Controller using Deep Learning" project signifies a substantive stride forward in human-computer interaction, introducing an innovative interface that promises a future where devices seamlessly respond to innate gestures, thereby rendering technology more accessible and user-focused.

Human-computer interaction in virtual reality environments for educational and business purposes

This paper examines the integration of Human-Computer Interaction (HCI) in Virtual Reality (VR) environments, with a focus on its application in educational and business domains. It explores the transformative potential of VR in enhancing user interface design, optimizing user experience, and integrating advanced technologies like natural language processing and gesture recognition. The research highlights VR's efficacy in improving learning outcomes and operational efficiencies, demonstrating significant advancements in immersive and intuitive interaction within virtual settings. The study underscores the critical role of user-centered design in VR systems and the impact of integrating natural language processing and gesture recognition technologies in creating more natural and intuitive user interactions. Through comprehensive analysis and case studies, the paper presents the latest developments in VR technology and its application, offering insights into the future of immersive digital environments in education and business sectors.

Gesture Recognition Based on Deep Learning: A Review

Gesture recognition is an important and inevitable technology in modern times, its appearance and improvement greatly improve the convenience of people's lives, but also enrich people's lives. It has a wide range of applications in various fields. In daily life, it can carry out human-computer interaction and the use of smart home. In terms of medical treatment, it can help patients to recover and assist doctors to carry out experiments. In terms of entertainment, it allows users to interact with the game in an immersive manner. This paper chooses three technologies that deep learning plays a more prominent role in gesture recognition, namely CNNs, LSTM and transfer learning based on deep learning. They each have their own advantages and disadvantages. Because of the different principles of use, different techniques have different roles, such as CNNs can carry out feature extraction, LSTM can deal with long time series, transfer learning can transfer what is learned from another task to this task. Select different practical technologies according to different application scenarios, and make improvements in real time in practical applications. Gesture recognition based on deep learning has the advantages of good accuracy, robustness and real-time implementation, but it also bears the disadvantages of huge economic and time costs and high hardware requirements. Despite some challenges, researchers continue to optimize and improve the technology, and believe that in the future, gesture recognition technology will be more mature and valuable.

Design and Development of Gesture Controlled Light Weight Robotic Arm

The Gesture-Controlled Robotic Arm revolutionizes cleaning and manufacturing industries by automating tasks and enhancing efficiency. Traditional methods often rely on manual labor, leading to inefficiencies and safety concerns. Our device utilizes advanced robotics and gesture recognition technology for precise control and maneuverability. It minimizes human intervention, reducing risks and enhancing workplace safety. With its adaptability to various environments, it performs tasks with unparalleled accuracy and versatility. Seamlessly integrating into existing workflows.

Real-time continuous gesture recognition system based on PSO-PNN

In the field of natural human–machine interaction, wearable gesture interaction technologies have received considerable attention, particularly continuous gesture (CG) recognition. However, CG faces several challenges, including the impact of motion characteristics on gesture recognition and performance that is not sufficiently robust. Traditional algorithms are highly dependent on samples, thus meeting the requirements of low sample volume and high accuracy simultaneously is challenging. To address these challenges, we propose a real-time CG recognition system based on particle swarm optimization (PSO) and probabilistic neural network (PNN). This system employs principal component analysis for signal dimensionality reduction to alleviate computational burden and uses K-means clustering and Pearson correlation coefficient to extract optimal features for gesture classification. In offline gesture recognition experiments involving six CGs, the algorithm achieved an accuracy rate of 97% with a training set of 300 samples and a runtime of just 31.25 ms. Compared to other five algorithms, the proposed algorithm improved accuracy by at least 9% and reduced the runtime by 40.475 ms. Moreover, gesture recognition experiments were conducted using different datasets, with the PSO-PNN algorithm achieving an average recognition rate of 90.17%, at least 9.84% higher than other algorithms. Finally, in experiments on online CG control for robots in complex environments, the PSO-PNN demonstrated real-time performance of 28.56 ms and a task completion rate of 90.67%, validating the feasibility of PSO-PNN. This research provides a substantial theoretical and technical foundation for the ongoing enhancement and application of CG recognition technology.

An Integrated Gesture Framework of Smart Entry Based on Arduino and Random Forest Classifier

Gesture-based systems have emerged as a prominent breakthrough in the field of smart access control, effectively integrating security measures with user comfort. This study presents a novel gesture detection framework for smart entry systems that combines the computational capabilities of a Random Forest Classifier with the practicality of Arduino-based hardware. Central to methodology is the utilization of MediaPipe, an advanced computer vision library, to extract hand motion landmarks from live video streams. The selected landmarks function as a comprehensive dataset for training a Random Forest Classifier, which has been specifically chosen due to its high level of accuracy and efficiency in managing intricate classification jobs. The model exhibits outstanding competence in the categorization of gestures in real-time, attaining high levels of accuracy that are crucial for ensuring dependable entrance control. The Arduino microcontroller plays a vital role in the execution of the entry mechanism as it serves as the intermediary between the gesture detection software and the tangible entry control hardware. The incorporation of gesture recognition technology facilitates a cohesive and prompt user experience, wherein identified motions are directly converted into input commands. The system's practical use is demonstrated through a series of detailed tests, which highlight its dependability and efficiency across diverse climatic circumstances. The findings underscore the system's capacity as a flexible and safe solution for contactless access in many environments, including both private homes and highly protected establishments. Furthermore, the study makes a substantial contribution to the larger domain of human-computer interaction by showcasing the practicality of advanced gesture detection systems in many everyday contexts. The suggested framework presents a novel approach to smart entry systems and also paves the way for further investigation in the domains of smart home automation and interactive systems. In these areas, gesture-based interfaces have the potential to deliver user experiences that are both intuitive and efficient.

Adaptive Trajectories’ Constant False Alarm Rate Mirror Filters and Elevation Angle Evaluation for Multiple-Input Multiple-Output Radar-Based Hand Gesture Recognition

Gesture recognition technology has been quickly developed in the field of human–computer interaction. The multiple-input multiple-output (MIMO) radar is popular in gesture recognition because of its notable spatial resolution. This work proposes a MIMO radar-based hand gesture recognition algorithm with low complexity. We leverage low-complexity adaptive signal processing to extract trajectory information and minimize noise to create a system that can be applied in real-world applications with small training datasets. First, a spectrum analysis is utilized on range-Doppler maps (RDMs), and a cell-averaging constant false alarm rate (CA-CFAR) with mirror filters is applied to improve the robustness of noise. Then, the features related to the distance, speed, direction, and elevation angle of the moving object are determined using the proposed adaptive signal analysis techniques. For classification, the random forest algorithm is implemented. The proposed system can precisely distinguish and identify eight gestures, including waving, moving to the left or right, patting, pushing, pulling, and rotating clockwise or anti-clockwise, with an accuracy of 95%. Experiments demonstrate the capability of the proposed hand gesture recognition system to classify different movements precisely.

Hand gesture recognition in natural human-computer interaction

This paper introduces the definition of Gesture recognition. First the article gives a precise definition of gesture recognition and explains the difference between gestures and postures, then it reveals technical difficulties of Gesture recognition. These technical difficulties include four aspects. After analyzing the technology and methods of Gesture recognition and the technical difficulties, it concretely expounds gesture recognition process based on data gloves, which is widely studied before and it also introduces the computer vision based research achievement which is currently becoming a research hotspot in this field. Then it also takes Kinect and HoloLens as examples to introduce specific practical cases of gesture recognition in wearable devices. Also it outlines the application of gesture recognition technology in human-computer interaction which include but not limited to smart terminal, Game control, Robot Control, Clinical and Health, Smart home, Sign Language Recognition, Vehicle system, Interactive entertainment. Finally it reach the conclusion that the biggest challenge researchers meet is to build a powerful framework to overcome the common problems with fewer constraints to provide reliable results. And sometimes researchers need to combine multiple methods for different complex environments.

Human-machine interactions based on hand gesture recognition using deep learning methods

Human interaction with computers and other machines is becoming an increasingly important and relevant topic in the modern world. Hand gesture recognition technology is an innovative approach to managing computers and electronic devices that allows users to interact with technology through gestures and hand movements. This article presents deep learning methods that allow you to efficiently process and classify hand gestures and hand gesture recognition technologies for interacting with computers. This paper discusses modern deep learning methods such as convolutional neural networks (CNN) and recurrent neural networks (RNN), which show excellent results in gesture recognition tasks. Next, the development and implementation of a human-machine interaction system based on hand gesture recognition is discussed. System architectures are described, as well as technical and practical aspects of their application. In conclusion, the article summarizes the research results and outlines the prospects for the development of hand gesture recognition technology to improve human-machine interaction. The advantages and limitations of the technology are analyzed, as well as possible areas of its application in the future.

The Implementation of a Gesture Recognition System with a Millimeter Wave and Thermal Imager.

During the COVID-19 pandemic, the number of cases continued to rise. As a result, there was a growing demand for alternative control methods to traditional buttons or touch screens. However, most current gesture recognition technologies rely on machine vision methods. However, this method can lead to suboptimal recognition results, especially in situations where the camera is operating in low-light conditions or encounters complex backgrounds. This study introduces an innovative gesture recognition system for large movements that uses a combination of millimeter wave radar and a thermal imager, where the multi-color conversion algorithm is used to improve palm recognition on the thermal imager together with deep learning approaches to improve its accuracy. While the user performs gestures, the mmWave radar captures point cloud information, which is then analyzed through neural network model inference. It also integrates thermal imaging and palm recognition to effectively track and monitor hand movements on the screen. The results suggest that this combined method significantly improves accuracy, reaching a rate of over 80%.

Research on gesture recognition algorithm based on MME-P3D.

A Multiscale-Motion Embedding Pseudo-3D (MME-P3D) gesture recognition algorithm has been proposed to tackle the issues of excessive parameters and high computational complexity encountered by existing gesture recognition algorithms deployed in mobile and embedded devices. The algorithm initially takes into account the characteristics of gesture motion information, integrating the channel attention (CE) mechanism into the pseudo-3D (P3D) module, thereby constructing a P3D-C feature extraction network that can efficiently extract spatio-temporal feature information while reducing the complexity of the algorithmic model. To further enhance the understanding and learning of the global gesture movement's dynamic information, a Multiscale Motion Embedding (MME) mechanism is subsequently designed. The experimental findings reveal that the MME-P3D model achieves recognition accuracies reaching up to 91.12% and 83.06% on the self-constructed conference gesture dataset and the publicly available Chalearn 2013 dataset, respectively. In comparison with the conventional 3D convolutional neural network, the MME-P3D model demonstrates a significant advantage in terms of parameter count and computational requirements, which are reduced by as much as 82% and 83%, respectively. This effectively addresses the limitations of the original algorithms, making them more suitable for deployment on embedded and mobile devices and providing a more effective means for the practical application of hand gesture recognition technology.

Bridging Communication Gaps: A Comprehensive Survey on Sign Language Hand Gesture Smart Recognition Technology

Abstract: Efficient communication is crucial in human interaction, yet individuals with language and hearing challenges encounter obstacles. This article centers on Nepali Sign Language (NSL) and introduces the Hand Gesture Smart Recognition initiative, with the goal of translating NSL into English. The research explores hand gesture recognition technology, emphasizing the cultural subtleties inherent in both vision-based and glove-based approaches. Recent advancements, such as the integration of compact models and CNNs, signify advancements in gesture recognition. The article offers valuable insights into the development and application of this technology, fostering optimism regarding the future potential of culturally suitable human-computer interaction technologies.

A Smart Device with Hand Gesture using Webcam

Abstract: Over time, Human-Computer Interaction (HCI) has experienced tremendous growth. An entirely new era in the AI domain of human-computer interaction was ushered in by hand gesture recognition technologies. One of the easiest and most natural forms of communication is hand gestures. It might be challenging for novice With hand gestures, the user can control cursor functions on the system. The input device for the suggested system is a camera. this application, HCI deployment for educational purposes will enter a completely new age.

Gestive: Evaluation of Multi-Class Classification Methods for Gesture Recognition to Improve Presentation Experience

Gesture recognition is able to transform how we interact with technology, including during presentations. We aim to classify slide-control gestures and create a model that allows users to add their personalized gestures to to control presentation com- mands. To avoid wasting time on capturing and training the model, we aim to construct a model with a short training time and high accuracy. We assessed the efficiency of various multi-class classification methods for gesture recognition in presentation settings. We compared K-Nearest Neighbor (KNN), Support Vector Machine (SVM), Decision Tree, Linear Discriminant Analysis (LDA), and Random Forest. The results demonstrated that KNN offers the shortest training time while maintaining high accuracy. The findings of this study provide a basis for future research on gesture recognition technology and its imple- mentation to improve presentation experience.