Gesture Images Research Articles

Gesture Recognition using Conditional Generative Adversarial Networks

Sign language is very useful for giving signs to communication partners, in this case, signs are not only for people with disabilities but can also be used by normal people. Even in children's or adult games, sign language is used as a language or a means of communication with one another. Recognition of sign language using a computer by doing several methods, because the computer does not recognize the image of a sign with a certain meaning. Therefore, it is necessary to train computers to recognize these signs. One of the fields that discusses gesture image recognition is the field of Computer Vision. Where the science of computer vision is able to process the image. In addition, in image processing, it is necessary to carry out deep learning processes such as Convolutional Neural Networks. In the Convolutional Neural Network algorithm, there are also many methods or architectures such as VGG16, VGG19, ResNet-50, DenseNet, Inception_V3, and many more. The use of architecture is used in accordance with existing needs. Therefore, the choice of architecture will determine the model is built or not to build from scratch, only transfer learning or pre-train. Pre-training is done by using the initial model and then using it only. Or do some training. The purpose of this study was to detect sign language using the Generative Adversarial Network (GAN). Actually, the Generative Adversarial Network method is widely used in making synthetic images, but this time the Generative Adversarial Network can also detect images from sign language.

Hand gesture recognition based on a Harris Hawks optimized Convolution Neural Network

Hand gestures are an effective method of communication, especially when we are communicating with people who cannot understand our spoken language. Furthermore, it is a key aspect to human–computer interaction. Understanding hand gestures is very important to ensure that listeners understand what speakers are attempting to communicate. Even though several researchers have proposed deep learning-based models for hand gesture recognition, the hyper-parameter tuning of these models is a relatively unexplored area. In this work, Convolutional Neural Networks (CNN) are used to classify hand gesture images. To tune the hyper-parameters of the CNN, a recently developed metaheuristic algorithm, namely, the Harris Hawks Optimization (HHO) algorithm, is used. Our in-depth comparative analysis proves that the proposed HHO-CNN hybrid model outperforms the existing models by attaining an Accuracy of 100%.

Real-Time Hand Gesture Recognition for Media Players

In this modern age of advancement in computing which has made the use of natural user interface much needed. Human gesture recognition is a mode of non-verbal interaction medium and can provide the most intuitive and natural way to interact with computer. The main aim is to make the interaction between human and computer as natural as the interaction between humans. The objective of this paper is to recognize the dynamic hand gesture images based on shapes and relative positions or direction of hand which is extracted from input video stream recording. These hand gestures can be used to recognition system to control multimedia applications running on computer using different gesture commands

Application of Deep Learning using Convolutional Neural Network (CNN) Algorithm for Gesture Recognition

Gesture recognition is a fascinating method of human-computer interaction that goes beyond traditional means such as keyboards, pointers, and joypads. In gesture recognition, Convolutional Neural Network (CNN) algorithms are utilized in Deep Learning to train models using datasets comprising gesture images. The training process involves pattern recognition and identification of crucial features from gesture images, followed by evaluation to measure the model's accuracy. Gesture recognition holds immense potential across various fields, including human-computer interaction, gaming, healthcare, and autonomous vehicles, and continues to be a focus of research and development in the future.

Sign Language Prediction Model using Convolution Neural Network.

The barrier between the hearing and the deaf communities in Kenya is a major challenge leading to a major gap in the communication sector where the deaf community is left out leading to inequality. The study used primary and secondary data sources to obtain information about this problem, which included online books, articles, conference materials, research reports, and journals on sign language and hand gesture recognition systems. To tackle the problem, CNN was used. Naturally captured hand gesture images were converted into grayscale and used to train a classification model that is able to identify the English alphabets from A-Z. Then identified letters are used to construct sentences. This will be the first step into breaking the communication barrier and the inequality. A sign language recognition model will assist in bridging the exchange of information between the deaf and hearing people in Kenya. The model was trained and tested on various matrices where we achieved an accuracy score of a 99% value when run on epoch of 10, the log loss metric returning a value of 0 meaning that it predicts the actual hand gesture images. The AUC and ROC curves achieved a 0.99 value which is excellent.

CNN Deep Learning with Wavelet Image Fusion of CCD RGB-IR and Depth-Grayscale Sensor Data for Hand Gesture Intention Recognition.

Pixel-based images captured by a charge-coupled device (CCD) with infrared (IR) LEDs around the image sensor are the well-known CCD Red–Green–Blue IR (the so-called CCD RGB-IR) data. The CCD RGB-IR data are generally acquired for video surveillance applications. Currently, CCD RGB-IR information has been further used to perform human gesture recognition on surveillance. Gesture recognition, including hand gesture intention recognition, is attracting great attention in the field of deep neural network (DNN) calculations. For further enhancing conventional CCD RGB-IR gesture recognition by DNN, this work proposes a deep learning framework for gesture recognition where a convolution neural network (CNN) incorporated with wavelet image fusion of CCD RGB-IR and additional depth-based depth-grayscale images (captured from depth sensors of the famous Microsoft Kinect device) is constructed for gesture intention recognition. In the proposed CNN with wavelet image fusion, a five-level discrete wavelet transformation (DWT) with three different wavelet decomposition merge strategies, namely, max-min, min-max and mean-mean, is employed; the visual geometry group (VGG)-16 CNN is used for deep learning and recognition of the wavelet fused gesture images. Experiments on the classifications of ten hand gesture intention actions (specified in a scenario of laboratory interactions) show that by additionally incorporating depth-grayscale data into CCD RGB-IR gesture recognition one will be able to further increase the averaged recognition accuracy to 83.88% for the VGG-16 CNN with min-max wavelet image fusion of the CCD RGB-IR and depth-grayscale data, which is obviously superior to the 75.33% of VGG-16 CNN with only CCD RGB-IR.

Real-Time Hand Gesture Recognition Using Fine-Tuned Convolutional Neural Network.

Hand gesture recognition is one of the most effective modes of interaction between humans and computers due to being highly flexible and user-friendly. A real-time hand gesture recognition system should aim to develop a user-independent interface with high recognition performance. Nowadays, convolutional neural networks (CNNs) show high recognition rates in image classification problems. Due to the unavailability of large labeled image samples in static hand gesture images, it is a challenging task to train deep CNN networks such as AlexNet, VGG-16 and ResNet from scratch. Therefore, inspired by CNN performance, an end-to-end fine-tuning method of a pre-trained CNN model with score-level fusion technique is proposed here to recognize hand gestures in a dataset with a low number of gesture images. The effectiveness of the proposed technique is evaluated using leave-one-subject-out cross-validation (LOO CV) and regular CV tests on two benchmark datasets. A real-time American sign language (ASL) recognition system is developed and tested using the proposed technique.

Static hand gesture recognition based on hierarchical decision and classification of finger features.

AbstractConsidering that the distinctions among static hand gestures are the difference between fingers sticking out, a method of grouping and classifying hand gestures step by step by using the information of the quantity, direction, position and shape of the outstretched fingers was proposed this paper. Firstly, the gesture region was segmented by using the skin color information of the hand, and the gesture direction was normalized by using the direction information of the gesture contour lines. Secondly, the finger was segmented one by one by using convex decomposition in the hand gesture image based on the convex characteristic of the gesture shape. Thirdly, the features of quantity, direction, position and shape of the segmented fingers were extracted. Lastly, a hierarchical decision classifier embedded with deep sparse autoencoders was constructed. The quantity of fingers was used to divide the gesture images into groups first, then the direction, position and shape features of the fingers were used to subdivide and recognize gestures within each group. The experimental results show that the proposed method is robust as lighting, direction and scale changes and significantly superior to the traditional method both in the recognition rate and the recognition stability.

A Low-cost Raspberry Pi-based Control System for Upper Limb Prosthesis

recent years, robots have been introduced in most factories. However, manual work still continues to be done in some places where giant robots cannot be installed. In particular, traditional Japanese crafts are done by hand, and people that engage in such crafts are called craftsmen. Generally, such artisans need years of training and cannot become experts right away. One of the problems these artisans face is the lack of successors. To address this challenge, this paper proposes a raspberry pi hardware based control method for a prosthetic hand using hand gestures from camera sensor, which will allow a prosthetic hand to learn the hand movements of the craftsmen and perform the crafts. The advantage of this is that there is no need for training, which usually takes years. To control the prosthetic hand, hand gestures are captured from a camera sensor, converted to HSV and binarized, and then classified into one of five gestures using a CNN implemented on the raspberry pi hardware. The recognized gesture is then relayed to the prosthetic hand to mimic the classified gesture. A dataset containing 2000 captured images of each gesture was created to evaluate the performance, and these gestures clearly define the closing and opening of the fingers. Using a 32×32 hand gesture image dataset captured from camera, we validated the trained CNN first in software for hand recognition without using Raspberry Pi, and achieved an accuracy of 99.63%, and then implemented on the raspberry pi, and performed real-time evaluation by recognizing five hand gestures captured from the camera sensor in real-time. Out of the hand gestures, four were correctly recognized. We presented the design of a low-cost prosthetic hand based on raspberry pi hardware, and evaluated its real-time hand gesture recognition. The evaluation result show that the proposed system is able to correctly recognize four hand gestures.

Research on Static Gesture Recognition Based on Deep Learning

Abstract With the continuous development and progress of the times, the ways of human-computer interaction have become more and more diverse. In order to reduce the spread of the new crown virus, gesture recognition has become a hot topic in the field of human-computer interaction in recent years. Traditional gesture recognition is affected by the environment and database, etc., with poor robustness and low accuracy. In order to improve the recognition rate of static gestures, this paper proposes to establish a deep learning model using CNN convolutional neural network, and a static gesture recognition method based on template matching. By establishing a palm template diagram, the gesture image to be recognized is matched with the template diagram based on the feature point, and the image is rotated after matching, and the template based on the grayscale value is matched again, so as to extract the gesture part. Through experimental proof, the algorithm can effectively improve the gesture recognition rate, the recognition accuracy rate reached 93.17%, and the recognition speed is faster.

A novel art gesture recognition model based on two channel region-based convolution neural network for explainable human-computer interaction understanding

The application development of hot technology is both an opportunity and a challenge. The vision-based gesture recognition rate is low and real-time performance is poor, so various algorithms need to be studied to improve the accuracy and speed of recognition. In this paper, we propose a novel gesture recognition based on two channel region-based convolution neural network for explainable human-computer interaction understanding. The input gesture image is extracted through two mutually independent channels. The two channels have convolution kernel with different scales, which can extract the features of different scales in the input image, and then carry out feature fusion at the fully connection layer. Finally, it is classified by the softmax classifier. The two-channel convolutional neural network model is proposed to solve the problem of insufficient feature extraction by the convolution kernel. Experimental results of gesture recognition on public data sets NTU and VIVA show that the proposed algorithm can effectively avoid the over-fitting problem of training models, and has higher recognition accuracy and stronger robustness than traditional algorithms.

Hand gesture recognition algorithm combining hand-type adaptive algorithm and effective-area ratio for efficient edge computing

Most existing gesture recognition algorithms have low recognition rates under rotation, translation, and scaling of hand images as well as different hand types. We propose a new hand gesture recognition algorithm that combines the hand-type adaptive algorithm and effective-area ratio based on feature matching. Samples are divided into several groups according to the subjects’ palm shapes and the algorithm is trained using self-collected data. The user’s hand type is paired with one of the sample libraries by the hand-type adaptive algorithm. To further improve the accuracy, the effective-area ratio of the gesture is calculated based on the minimum bounding rectangle, and the preliminary gesture is recognized by the effective-area ratio feature method. The results of experiments demonstrate that the proposed algorithm could accurately recognize gestures in real time and exhibits good adaptability to different hand types. The overall recognition rate is over 94%. The recognition rate still exceeds 93% when hand gesture images are rotated, translated, or scaled.

High Inclusiveness and Accuracy Motion Blur Real-Time Gesture Recognition Based on YOLOv4 Model Combined Attention Mechanism and DeblurGanv2

The combination of gesture recognition and aerospace exploration robots can realize the efficient non-contact control of the robots. In the harsh aerospace environment, the captured gesture images are usually blurred and damaged inevitably. The motion blurred images not only cause part of the transmitted information to be lost, but also affect the effect of neural network training in the later stage. To improve the speed and accuracy of motion blurred gestures recognition, the algorithm of YOLOv4 (You Only Look Once, vision 4) is studied from the two aspects of motion blurred image processing and model optimization. The DeblurGanv2 is employed to remove the motion blur of the gestures in YOLOv4 network input pictures. In terms of model structure, the K-means++ algorithm is used to cluster the priori boxes for obtaining the more appropriate size parameters of the priori boxes. The CBAM attention mechanism and SPP (spatial pyramid pooling layer) structure are added to YOLOv4 model to improve the efficiency of network learning. The dataset for network training is designed for the human–computer interaction in the aerospace space. To reduce the redundant features of the captured images and enhance the effect of model training, the Wiener filter and bilateral filter are superimposed on the blurred images in the dataset to simply remove the motion blur. The augmentation of the model is executed by imitating different environments. A YOLOv4-gesture model is built, which collaborates with K-means++ algorithm, the CBAM and SPP mechanism. A DeblurGanv2 model is built to process the input images of the YOLOv4 target recognition. The YOLOv4-motion-blur-gesture model is composed of the YOLOv4-gesture and the DeblurGanv2. The augmented and enhanced gesture data set is used to simulate the model training. The experimental results demonstrate that the YOLOv4-motion-blur-gesture model has relatively better performance. The proposed model has the high inclusiveness and accuracy recognition effect in the real-time interaction of motion blur gestures, it improves the network training speed by 30%, the target detection accuracy by 10%, and the value of mAP by about 10%. The constructed YOLOv4-motion-blur-gesture model has a stable performance. It can not only meet the real-time human–computer interaction in aerospace space under real-time complex conditions, but also can be applied to other application environments under complex backgrounds requiring real-time detection.

How does hand gestures in videos impact social media engagement - Insights based on deep learning

With the fast improvement and development in deep learning and computer vision, the interaction between humans and computers is becoming immense. This research work aims at recognizing hand gestures performed in TEDx videos and analyse the relationship between gestures and user engagement. Here we propose a technique based on deep learning, Convolutional Neural Network (CNN), to recognize hand gestures from a video or image input. ResNeXt-101 model is used for the classification of hand gestures. Here we also used images from the Twenty Bn dataset for some gestures while we also collected gesture images from TEDx videos. Our experiments obtained high accuracy for gesture recognition on training as 95% to 99% and 94.35% for testing. Firstly a gesture is identified in each frame and then classified. We also created a count function that helped us count the number of times a gesture was performed and help us analyse these talks in a better way. Two experimental studies were carried out to analyse viewer engagement: one, the effect of suitable gestures on the viewer count, and secondly, suitable gestures on the sentiment of the viewer. Interesting results are observed that suitable gestures from talkers have an impact on increasing positive review and viewer count.

Image Segmentation Method for an Illumination Highlight Region of Interior Design Effects Based on the Partial Differential Equation

The saliency calculation model based on the principle of partial differential equations sometimes highlights areas with high contrast in the background, and the salient targets obtained occasionally have holes. The above problems can be solved by combining the improved convex hull calculation center saliency map. This paper designs a single-target color image segmentation algorithm based on partial differential equations. First, we calculate the basic saliency map according to the uniqueness of the color and the spatial distribution of the color; second, we then use the superpixel to improve the convex hull and calculate the central saliency map according to the principle; finally, the basic saliency map and the central saliency map are calculated. The weighted fusion is used to obtain the comprehensive saliency map, and the threshold method is used to segment the comprehensive saliency map to obtain the final target image. This paper designs an evaluation standard suitable for the segmentation of the illuminated highlight area of the effect image. It compares the experimental results of the segmentation method in this paper with the SLIC (Simple Linear Iterative Clustering) method and the traditional superpixel method to segment the illuminated highlight area. The segmentation method is applied to the image enhancement experiment. Based on the fuzzy means clustering algorithm, a fuzzy clustering objective function including brightness, color, and distance parameters is designed, which improves the weight of the brightness value in the clustering and improves the edge fit of the segmentation of the lighting highlight area of the rendering. The segmentation method produced by combining the clustering method with the superpixel biased clustering method can improve the output effect of the illuminated highlight area of the effect image after segmentation. We perform color equalization processing on the image to be segmented to reduce the impact of light, then set the closed value of the brightness information component, perform segmentation judgment, and expand the long and short axes of the ellipse model in the high-brightness area to further reduce the impact of light. The experimental results prove that the above method has a better segmentation effect than the traditional ellipse model and can accurately segment the gesture image. Compared with the existing mainstream saliency calculation models, this algorithm is closer to the true value image in terms of visual effects and has obvious advantages in terms of accuracy.

Using Deep Learning Technology to Realize the Automatic Control Program of Robot Arm Based on Hand Gesture Recognition

In this study, the robot arm control, computer vision, and deep learning technologies are combined to realize an automatic control program. There are three functional modules in this program, i.e., the hand gesture recognition module, the robot arm control module, and the communication module. The hand gesture recognition module records the user’s hand gesture images to recognize the gestures’ features using the YOLOv4 algorithm. The recognition results are transmitted to the robot arm control module by the communication module. Finally, the received hand gesture commands are analyzed and executed by the robot arm control module. With the proposed program, engineers can interact with the robot arm through hand gestures, teach the robot arm to record the trajectory by simple hand movements, and call different scripts to satisfy robot motion requirements in the actual production environment.

Indian Sign Language Recognition Using Canny Edge Detection

n our society, it is very difficult for hearing impaired and speech impaired people to communicate with ordinary people. They use sign languages to communicate, which use visually transmitted sign patterns, generally includes hand gestures. Sign languages being difficult to learn and non-universal, there is a barrier of communication between the hearing impaired and ordinary people. To break this barrier a system is required that can convert sign language to voice and vice versa in real-time. Here, we propose a real-time two-way system, for communication between hearing-impaired and normal people, which converts the Indian Sign Language (ISL) letters into equivalent alphabet letters and vice versa. In the proposed system, using a camera, images of ISL hand gestures are captured. Then Image pre-processing is done so that these images are ready for feature extraction. Here, a novel approach of using the Canny Edge Detection Algorithm. Once the necessary details are extracted from the image, it is matched with the data set, which is classified using Convolutional Neural Network, and the corresponding text is generated. This text is converted into a voice. Similarly, using a microphone, the voice input of an ordinary person is captured and converted into text. This text is then matched with the data set and a corresponding sign is generated. This system reduces the gap in communication between hearing-impaired and ordinary people. Our method provides 98 % accuracy for the 35 alphanumeric gestures of ISL

Hand Gesture Recognition By Deep Convolutional Neural Network

Today the hand gesture is increasingly in the human-computer interactions. In all project, there is a minimum number of the process involved in the hands only and maximum contains the body, background, etc., In this paper, it shows that the location of the hand, hand gesture, and applies it in home and also the interaction between the system and the users. Hand gesture means that it is the form of sign language in the form of passing the signals and expressions. The movement occurred in the hand is called the hand gesture which can be used in hearing loss peoples. The processing of the hand gesture is that it shares the communication to others with the help of hands. The different symbols and numbers are forming with the hands and the signs form a group of words that can be understood by other people. There are two types of hand gestures like a glove based and vision-based.In this paper, a new approach called deep convolutional neural networks, which used in hand gestures to classify the images held in the hand gesture. The deep convolutional hand gesture can be used to find a more number of hand gestures. The hand gestures require more time for processing. The convolutional neural network can be used for saving the timing and reducing the time used in the separation of things or images. Finding the state of hand gesture, which will also, needs more time. The deep CNN provides a faster process in that. Using the deep CNN method, provide the system with more accuracy, specificity, sensitivity compared to the other systems. The proposed method as CNN, which separates the different sections, provides good accuracy, specificity, sensitivity from the dataset. The system process ends until the hand gesture image is leaving from the camera.

Contactless interactive control technology with a switching system based on Butterworth filter and modified strong tracking Kalman filter

In the human–computer interaction field, a contactless interaction with large screens through gestures is very representative, and the recognition and filtering of gesture images are very important tasks. Aiming at solving the problems of interference and positioning drift of three-dimensional lidar sensors, this article proposes a contactless interactive control system based on switching filtering algorithm, which selects the Butterworth filtering and the modified strong tracking Kalman filter to be used in the filtering process. The proposed interactive system extracts and optimizes user gestures, maps the gestures to the screen, simulates mouse operations, and enables operations such as selection, sliding, zooming in and out, and others. This switching filtering algorithm effectively solves the accuracy problem of a single filtering algorithm and the rapidity of complex filtering algorithms in the signal processing step, and greatly improves the interaction accuracy without sacrificing too much processing time. The experimental results show that by applying the proposed switching filtering algorithm to a contactless human–computer interaction system, the system can achieve smooth gesture interaction. The proposed system can perform real-time interaction with multiple people, which fully verifies the effectiveness and superiority of the proposed algorithm.

Sewing gesture image detection method based on improved SSD model

Sewing gesture image detection method based on improved SSD model