Network For Hand Gesture Recognition Research Articles

Adaptive shift graph convolutional neural network for hand gesture recognition based on 3D skeletal similarity

Adaptive shift graph convolutional neural network for hand gesture recognition based on 3D skeletal similarity

A deep neural network for hand gesture recognition from RGB image in complex background

A deep neural network for hand gesture recognition from RGB image in complex background

Mask-guided multiscale feature aggregation network for hand gesture recognition

Hand gesture recognition from images is a longstanding computer vision task that can be used to build a potential bridge for human-computer interaction and sign language translation. For number of methods proposed for hand gesture recognition (HGR); however, difficult scenarios such as different scales of hand gestures and complex backgrounds exist, making them less effective. In this paper, we propose an end-to-end multiscale feature learning network for HGR, which consists of a CNN-based backbone network, a feature aggregation pyramid network (FAPN) embedded with a two-stage expansion-squeeze-aggregation (ESA) module, and three task-specific prediction branches. First, the backbone network extracts multiscale features from the original hand gesture images. Furthermore, the FAPN embedded with two-stage ESA extensively exploits multiscale feature information and learns hand gesture-specific features at different scales. Then, the mask loss guides the network to locate hand-specific regions during the training stage, and finally, the classification and regression branches output the category and location of a hand gesture during the model training and prediction. The experimental results on two publicly available datasets show that the proposed method outperforms most state-of-the-art HGR methods.

An enhanced artificial neural network for hand gesture recognition using multi-modal features

ABSTRACT The development of automatic application fields enhanced the advancement in Hand Gesture Recognition (HGR), which enhances human-computer interaction. In particular, hand gestures find a valuable application in hand games, virtual reality, and gesture-based signalling systems. Thus, detecting and recognising hand gestures gain technical significance. Although existing models highlight remarkable recognition outcomes, handling dynamic gestures is a challenging task. Hence, in this research, an enhanced flashfly optimisation-based artificial neural network is developed for the effective identification of hand gestures, and to overcome the existing challenges, the enhanced flash fly optimisation-based ANN technique is proposed. The main objective of the optimisation strategy is to effectively tune the parameters involved in the artificial neural network (ANN) by the attained optimal solution. The optimised ANN classifier is well trained by the pre-processed statistical and the Residual neural Networks (ResNet) features for consistently identifying hand movements. The attained performance of the enhanced flash fly optimisation-based ANN classifier during the representation of hand gestures based on the precision, recall, and F1 score is 0.940 %, 0.791 %, and 0.927 % for the training percentages 50 and 90. Hence, it is illustrated that the proposed HGR model exceeds the competent technique, demonstrating its effectiveness.

Multistage Spatial Attention-Based Neural Network for Hand Gesture Recognition

The definition of human-computer interaction (HCI) has changed in the current year because people are interested in their various ergonomic devices ways. Many researchers have been working to develop a hand gesture recognition system with a kinetic sensor-based dataset, but their performance accuracy is not satisfactory. In our work, we proposed a multistage spatial attention-based neural network for hand gesture recognition to overcome the challenges. We included three stages in the proposed model where each stage is inherited the CNN; where we first apply a feature extractor and a spatial attention module by using self-attention from the original dataset and then multiply the feature vector with the attention map to highlight effective features of the dataset. Then, we explored features concatenated with the original dataset for obtaining modality feature embedding. In the same way, we generated a feature vector and attention map in the second stage with the feature extraction architecture and self-attention technique. After multiplying the attention map and features, we produced the final feature, which feeds into the third stage, a classification module to predict the label of the correspondent hand gesture. Our model achieved 99.67%, 99.75%, and 99.46% accuracy for the senz3D, Kinematic, and NTU datasets.

A Lightweight Network Deployed on ARM Devices for Hand Gesture Recognition

Hand gestures are a well-known and straightforward method of human-computer interaction. The majority of the study focused on hand gesture recognition. However, little work has been done to develop a complete set of gesture recognition applications. With the improvement of model feature extraction ability and the increase in the number of model parameters, it is becoming more challenging to achieve a small memory footprint on mobile devices based on an ARM architecture or CPU devices based on x86 architecture. However, these existing methods are heavy, requiring more memory and inference time. The execution of memory-efficient CNNs without compromising accuracy has been a challenge, especially when the inference has to be performed on an edge computing device in real time. Therefore, we propose a lightweight network for hand gesture recognition (LHGR-Net) and deploy it on a Raspberry Pi. LHGR-Net consists of three main parts: the base network structure, the multiscale structure (MSS), and the lightweight attention structure (LAS). We present pre-trained weights that are learned from other data to initialize the network structure. In addition, the LHGR-Net model was made to be deployed on a Raspberry Pi, and a deployed model can be used to control home appliances. Extensive experiments show that our method achieves almost as good as state-of-the-art performance in hand gesture recognition and running time.

LSTM Recurrent Neural Network for Hand Gesture Recognition Using EMG Signals

Currently, research on gesture recognition systems has been on the rise due to the capabilities these systems provide to the field of human–machine interaction, however, gesture recognition in prosthesis and orthesis has been carried out through the use of an extensive amount of channels and electrodes to acquire the EMG (Electromyography) signals, increasing the cost and complexity of these systems. The scientific literature shows different approaches related to gesture recognition based on the analysis of EMG signals using deep learning models, highlighting the recurrent neural networks with deep learning structures. This paper presents the implementation of a Recurrent Neural Network (RNN) model using Long-short Term Memory (LSTM) units and dense layers to develop a gesture classifier for hand prosthesis control, aiming to decrease the number of EMG channels and the overall model complexity, in order to increase its scalability for embedded systems. The proposed model requires the use of only four EMG channels to recognize five hand gestures, greatly reducing the number of electrodes compared to other approaches found in the literature. The proposed model was trained using a dataset for each gesture EMG signals, which were recorded for 20 s using a custom EMG armband. The model reached an accuracy of to 99% for the training and validation stages, and an accuracy of 87 ± 7% during real-time testing. The results obtained by the proposed model establish a general methodology for the reduction of complexity in the recognition of gestures intended for human.machine interaction for different computational devices.

Touchless Control of Heavy Equipment Using Low-Cost Hand Gesture Recognition

Human-machine interaction using remote hand gestures is becoming increasingly prevalent across various industries. However, their potential application to heavy construction equipment is often overlooked. This article presents a robust and inexpensive hand gesture recognition system that was implemented and tested on a robotic 1-tonne wheel loader. The system uses an RGB camera paired with a laptop to process, in real time, hand gestures to control the loader. We first design four unique gestures for controlling the loader and then collect 26,000 images to train and test a neural network for hand gesture recognition. Our system uses robust landmark detection using an off-the-shelf system prior to gesture recognition. We successfully controlled the loader to excavate in a rock pile by using the proposed hand gesture recognition system.

Content-Adaptive and Attention-Based Network for Hand Gesture Recognition

For hand gesture recognition, recurrent neural networks and 3D convolutional neural networks are the most commonly used methods for learning the spatial–temporal features of gestures. The calculation of the hidden state of the recurrent neural network at a specific time is determined by both input at the current time and the output of the hidden state at the previous time, therefore limiting its parallel computation. The large number of weight parameters that need to be optimized leads to high computational costs associated with 3D convolution-based methods. We introduced a transformer-based network for hand gesture recognition, which is a completely self-attentional architecture without any convolution or recurrent layers. The framework classifies hand gestures by focusing on the sequence information of the whole gesture video. In addition, we introduced an adaptive sampling strategy based on the video content to reduce the input of gesture-free frames to the model, thus reducing computational consumption. The proposed network achieved 83.2% and 93.8% recognition accuracy on two publicly available benchmark datasets, NVGesture and EgoGesture datasets, respectively. The results of extensive comparison experiments show that our proposed approach outperforms the existing state-of-the-art gesture recognition systems.

HyFiNet: Hybrid feature attention network for hand gesture recognition

HyFiNet: Hybrid feature attention network for hand gesture recognition

A Deep Neural Network for Hand Gesture Recognition from Rgb Image in Complex Background

A Deep Neural Network for Hand Gesture Recognition from Rgb Image in Complex Background

ExtriDeNet: an intensive feature extrication deep network for hand gesture recognition

ExtriDeNet: an intensive feature extrication deep network for hand gesture recognition

Dynamic hand gesture recognition based on short-term sampling neural networks

Hand gestures are a natural way for human-robot interaction. Vision based dynamic hand gesture recognition has become a hot research topic due to its various applications. This paper presents a novel deep learning network for hand gesture recognition. The network integrates several well-proved modules together to learn both short-term and long-term features from video inputs and meanwhile avoid intensive computation. To learn short-term features, each video input is segmented into a fixed number of frame groups. A frame is randomly selected from each group and represented as an RGB image as well as an optical flow snapshot. These two entities are fused and fed into a convolutional neural network (ConvNet) for feature extraction. The ConvNets for all groups share parameters. To learn longterm features, outputs from all ConvNets are fed into a long short-term memory (LSTM) network, by which a final classification result is predicted. The new model has been tested with two popular hand gesture datasets, namely the Jester dataset and Nvidia dataset. Comparing with other models, our model produced very competitive results. The robustness of the new model has also been proved with an augmented dataset with enhanced diversity of hand gestures.

Feature based weighted neural network for hand gesture recognition

Feature based weighted neural network for hand gesture recognition

Testing artificial neural network for hand gesture recognition

Neural networks are well applicable for gesture recognition. In this article we want to present the result of an artificial feed forward network for a simplified hand gesture recognition problem. In neural networks, the learning algorithm is very important because the performance of neural network depends on it. One of the most known learning algorithm is the backpropagation. There are some mathematical software which provides acceptable result for a given problem by a backpropagation based network, but in some cases a high-level programming language implemented program can provide better solution. The main topics of the article cover the structure of the test environment, the mathematical background of the implemented methods, some programming remarks and the test results.