Real-time Dynamic Gesture Recognition Research Articles

Real-Time Dynamic Gesture Recognition Algorithm Based on Adaptive Information Fusion and Multi-Scale Optimization Transformer

Gesture recognition is a popular technology in the field of computer vision and an important technical mean of achieving human-computer interaction. To address problems such as the limited long-range feature extraction capability of existing dynamic gesture recognition networks based on convolutional operators, we propose a dynamic gesture recognition algorithm based on spatial pyramid pooling Transformer and optical flow information fusion. We take advantage of Transformer’s large receptive field to reduce model computation while improving the model’s ability to extract features at different scales by embedding spatial pyramid pooling. We use the optical flow algorithm with the global motion aggregation module to obtain an optical flow map of hand motion, and to extract the key frames based on the similarity minimization principle. We also design an adaptive feature fusion method to fuse the spatial and temporal features of the dual channels. Finally, we demonstrate the effectiveness of model components on model recognition enhancement through ablation experiments. We conduct training and validation on the SCUT-DHGA dynamic gesture dataset and on a dataset we collected, and we perform real-time dynamic gesture recognition tests using the trained model. The results show that our algorithm achieves high accuracy even while keeping the parameters balanced. It also achieves fast and accurate recognition of dynamic gestures in real-time tests.

Real-Time Gesture Recognition in the View of Repeating Characteristics of Sign Languages

Recognizing dynamic gestures from a continuous data stream has been treated as a difficult task. One challenge is the gesture spotting problem, or segmentation of a gesture pattern from a data stream containing consecutive gestures. Although various types of gesture spotting strategies have been introduced so far, all methods has its own limitations to be applied to real world situations such as sign language conversation. One of the conditions that must be satisfied for practical applications is to detect repetitive gestures for emphasizing a certain meaning.In the view of gesture spotting, repetitive gestures are hard to be detected because similar starting and ending moments are repeated without constraints of the number of repetitions. We introduce a data glove-based real-time dynamic gesture recognition system that covers both repetitive and nonrepetitive gestures. The proposed system consists of three steps: Gesture spotting, gesture sequence compression, and gesture recognition. For gesture spotting, we define a quantity termed as a gesture progress sequence, which quantifies a progress of a gesture using numbers from 0 to 1. Gesture sequence compression removes variations including the number of repetitions that seek to impart the same message. At the gesture recognition step, the compressed gesture patterns are classified. The proposed system was evaluated using 17 American sign language (ASL) gestures and some ASL sentences.

A 28 nm, 397 [formula omitted]W real-time dynamic gesture recognition chip based on RISC-V processor

This paper presents a low-power programmable dynamic gesture recognition chip based on a RISC-V processor using RGB images. The chip uses two gesture recognition algorithms to recognize the most commonly used dynamic gestures (up, down, left, and right) with any shape of a hand at the distance of 10 cm ∼ 90 cm and speed of 10 cm/s ∼ 150 cm/s. The chip is fabricated in a 28 nm process and occupies an area of 640μm x 640μm. The accuracy reaches 93% ∼ 99% at complex backgrounds with 320x240 RGB image. It operates either in the recognition mode or in the standby mode that can be automatically switched to the other. The power consumption of the recognition mode is 397 μW at 0.584 V supply voltage, 25 MHz clock, and 30 FPS. And the power consumption of the standby mode at room temperature is 78.3 μW.

Mechanical arm teleoperation control system by dynamic hand gesture recognition based on kinect device

The research achieved to control the mechanical arm by using real-time dynamic gesture recognition based on Kinect. It uses the unmarked gesture segmentation algorithm based on the palm neighbourhood and the threshold detection algorithm based on the palmar contour to identify the operator's gestures and moving trajectories, and to convert it into the specific action of the mechanical arm. Through wireless network, this system sends control instructions to the mechanical arm to realise teleoperation control. The system also provides video feedback of mechanical arm operation site. It improves the teleoperation telepresence and interactivity and avoids operation error such as grasp nothing or fall. The results of the experiment indicate that the operation of the gesture control system is simple and easy, the response of mechanical arm is quick and accurate and the human–machine interaction is intuitive and friendly.

Dynamic gesture recognition by directional pulse coupled neural networks for human-robot interaction in real time

Dynamic gesture recognition is a major topic for most real-time human-robot interaction applications. Recently, increasingly people have focused on dynamic gesture recognition based on 3D representation. In this paper, the gesture recognition is converted into the shortest path problem by transforming the feature matrix to an undirected graph and a novel dynamic gesture recognition algorithm of directional pulse coupled neuron network (DPCNN) is proposed for real time human-robot interactions. The DPCNN can select the firing direction by giving different excitations to neighbor neurons and reduce the effects of useless neurons. Furthermore, to reduce the recognition time, an early gesture recognition method based on the adaptive window is introduced to recognize the unfinished gestures. DPCNN reduces the computation time and achieves a high recognition rate on three public datasets compared with other algorithms which improves the efficiency of real-time dynamic gesture recognition and ensures a friendly experience for human-robot interactions.

Real-time dynamic gesture recognition and hand servo tracking using PTZ camera

A technology of real-time dynamic gesture recognition and hand tracking using a Pan-Tilt-Zoom (PTZ) camera was presented in this study. It was aimed to achieve robust scheme that stably recognized simple hand gestures and tracked the hand by means of a PTZ camera to keep the fingertip remaining in the center of the camera. For this purpose, the hand region was initially segmented in a cluttered environment using skin color segmentation in YCbCr color space to get the silhouette of the hand. Furthermore, the Monte Carlo Sampling method was used to estimate the Cubic Bezier curves best fitted to the sub contour points centralized in each contour point, and the fingertips were detected by combining the local maximums of a cumulative curvature with detection of convex defects. After that, feature triangle analysis was utilized to achieve dynamic recognition of simple gestures including “right click down” and “right click up”. Finally, the PTZ camera was driven by the algorithm to achieve servo tracking with the target fingertip when the gesture “right click down” was detected. As is shown by the experimental results, the proposed approach recognized the dynamic gestures and located the fingertips’ positions precisely, and realized the follow-up servo tracking using PTZ camera in real-time.

Real-Time Musical Conducting Gesture Recognition Based on a Dynamic Time Warping Classifier Using a Single-Depth Camera

Gesture recognition is a human−computer interaction method, which is widely used for educational, medical, and entertainment purposes. Humans also use gestures to communicate with each other, and musical conducting uses gestures in this way. In musical conducting, conductors wave their hands to control the speed and strength of the music played. However, beginners may have a limited comprehension of the gestures and might not be able to properly follow the ensembles. Therefore, this paper proposes a real-time musical conducting gesture recognition system to help music players improve their performance. We used a single-depth camera to capture image inputs and establish a real-time dynamic gesture recognition system. The Kinect software development kit created a skeleton model by capturing the palm position. Different palm gestures were collected to develop training templates for musical conducting. The dynamic time warping algorithm was applied to recognize the different conducting gestures at various conducting speeds, thereby achieving real-time dynamic musical conducting gesture recognition. In the experiment, we used 5600 examples of three basic types of musical conducting gestures, including seven capturing angles and five performing speeds for evaluation. The experimental result showed that the average accuracy was 89.17% in 30 frames per second.

A Posteriori Processing Optimization Method for Gesture Interaction in Aseptic Operation Room

Gesture recognition plays an important role in the applications of human-centered smart technologies. This paper designs a gesture recognition approach in a special human-computer interaction environment. Generally, in an aseptic operation room, doctors cannot access the patient’s image data directly, and they have to let the nurse operate the system under their instructions. After the nurses finish their task, they also need to re-dress the surgical gown, disinfect their hands, then they can go back to the operating room. If the doctor could access the image directly by gestures during the operation, they would be able to save a lot of time and work more efficiently. Although some template matching algorithms, such as dynamic time warping (DTW), have been widely used in gesture recognition, those traditional methods are not accurate enough, and they are unable to identify continuous repetitive gestures. This paper proposes an improved DTW method based on posteriori processing, which achieved some key operations, including parameter regulations, invalid area determination, and static gesture processing. This method will address the problem of continuous repetitive gesture recognition, with higher accuracy. The experimental results show that the method can improve the accuracy of recognition by over 40%, and it can also perform real-time dynamic gesture recognition tasks effectively.

A Fuzzy Framework for Real-Time Gesture Spotting and Recognition

A vital requirement of any recognition system claiming to be real time is the capability to perform feature extraction in real time. In this paper, we propose an innovative fuzzy approach for real-time dynamic gesture recognition and spotting, where a compact local descriptor is designed to model moving gesture skeletons as a time series of fuzzy statistical features. Then, a set of one-vs-rest SVMs is trained on these features for gesture recognition and spotting. In this approach, the meaningful hand movements are successfully spotted while concurrently removing unintentional hand movements from an input video sequence. When evaluated on a gesture data set incorporating a relatively large and diverse collection of video data, the method proposed yields promising results that compare very favorably with those reported in the literature, while retaining real-time performance.