Gesture Features Research Articles

Gesture Recognition with Residual LSTM Attention Using Millimeter-Wave Radar

Gesture recognition technology based on millimeter-wave radar can recognize and classify user gestures in non-contact scenarios. To address the complexity of data processing with multi-feature inputs in neural networks and the poor recognition performance with single-feature inputs, this paper proposes a gesture recognition algorithm based on ResNet Long Short-Term Memory with an Attention Mechanism (RLA). In the aspect of signal processing in RLA, a range–Doppler map is obtained through the extraction of the range and velocity features in the original mmWave radar signal. Regarding the network architecture in RLA, the relevant features of the residual network with channel and spatial attention modules are combined to prevent some useful information from being neglected. We introduce a residual attention mechanism to enhance the network’s focus on gesture features and avoid the impact of irrelevant features on recognition accuracy. Additionally, we use a long short-term memory network to process temporal features, ensuring high recognition accuracy even with single-feature inputs. A series of experimental results show that the algorithm proposed in this paper has higher recognition performance.

Reproducing the caress gesture with an anthropomorphic robot: a feasibility study

Social robots have been widely used to deliver emotional, cognitive and social support to humans. The exchange of affective gestures, instead, has been explored to a lesser extent, despite phyisical interaction with social robots could provide the same benefits as human-human interaction. Some studies that explored the touch and hugs gestures were found in literature, but there are no studies that investigate the possibility of delivering realistic caress gestures, which are, in turn, the easiest affective gestures that could be delivered with a robot. The first objective of this work was to study the kinematic and dynamic features of the caress gesture by conducting experimental acquisitions in which ten healthy volunteers were asked to caress the cheek of a mannequin in two conditions, i.e. standing and sitting. Average motion and force features were then analyzed and used to generate a realistic caress gesture with an anthropomorphic robot, with the aim of assessing the feasibility of reproducing the caress gesture with a robotic device. In addition, twenty-six healthy volunteers evaluated the anthropomorphism and perceived safety of the reproduced affective gesture by answering the Godspeed Questionnaire Series and a list of statements on the robot motion. The gesture reproduced by the robot was similar to the caress gesture performed by healthy volunteers both in terms of hand trajectory and orientation, and exchanged forces. Overall, volunteers perceived the robot motion as safe and positive emotions were elicited. The proposed approach could be adapted to humanoid robots to improve the perceived anthropomorphism and safety of the caress gesture.

Few-shot Learning for Sign Language Recognition with Embedding Propagation

Sign language is a primary channel for the deaf and hard-hearing to communicate. Sign language consists of many signs with different variations in hand shapes, motion patterns, and positioning of hands, faces, and body parts. This makes sign language recognition (SLR) a challenging field in computer vision research. This paper tackles the problem of few-shot SLR, where models trained on known sign classes are utilized to recognize instances of unseen signs with only a few examples. In this approach, a transformer encoder is employed to learn the spatial and temporal features of sign gestures, and an embedding propagation technique is used to project these features into the embedding space. Subsequently, a label propagation method is applied to smooth the resulting embeddings. The obtained results demonstrate that combining embedding propagation with label propagation enhances the performance of the SLR system and achieved an accuracy of 76.6%, which surpasses the traditional few-shot prototypical network's accuracy of 72.4%.

Social negotiation and “accents” in Western lowland gorillas’ gestural communication

Recent findings on chimpanzee infants’ gestural development show that they use some gesture types flexibly and adjust them depending on their interaction partner and social context, suggesting that gestural communication is partly learnt and partly genetically determined. However, how gesture types are shaped by social and demographic factors remains unclear. We addressed this question by focusing on gesture type morphology and conducted a fined-grained analysis of gestural form during intraspecific social-play interactions in two captive groups of Western lowland gorillas (Gorilla gorilla gorilla). We focused on the most frequent gesture types (beat chest, slap body, slap ground and touch body) produced by subadults (infants, juveniles and adolescents). We considered twelve morphological gesture characteristics (e.g., horizontal and vertical hand trajectories, fingers flexion and spread). Our multifactorial investigation shows that morphological characteristics of distinct gesture types can be shaped by social factors, namely signaller’s sociodemographic characteristics (group and kinship), signaller’s behavioural characteristics (body posture) and context-related characteristics (recipient’s sex, attentional state and position in the signaller’s visual field). We nurtured the lively debate concerning gesture origins by revealing the existence of “accents” in non-verbal communication and the highly variable adjustment of gestural form to different conspecifics and interactional characteristics, which supports the revised social negotiation hypothesis.

Multimodal Emotion Recognition Based on Facial Expressions, Speech, and Body Gestures

The research of multimodal emotion recognition based on facial expressions, speech, and body gestures is crucial for oncoming intelligent human–computer interfaces. However, it is a very difficult task and has seldom been researched in this combination in the past years. Based on the GEMEP and Polish databases, this contribution focuses on trimodal emotion recognition from facial expressions, speech, and body gestures, including feature extraction, feature fusion, and multimodal classification of the three modalities. In particular, for feature fusion, two novel algorithms including supervised least squares multiset kernel canonical correlation analysis (SLSMKCCA) and sparse supervised least squares multiset kernel canonical correlation analysis (SSLSMKCCA) are presented, respectively, to carry out efficient facial expression, speech, and body gesture feature fusion. Different from the traditional multiset kernel canonical correlation analysis (MKCCA) algorithms, our SLSKMCCA algorithm is a supervised version and is based on the least squares form. The SSLSKMCCA algorithm is implemented by the combination of SLSMKCCA and a sparse item (L1 Norm). Moreover, two effective solving algorithms for SLSMKCCA and SSLSMKCCA are presented in addition, which use the alternated least squares and augmented Lagrangian multiplier methods, respectively. The extensive experimental results on the popular public GEMEP and Polish databases show that the recognition rate of multimodal emotion recognition is superior to bimodal and monomodal emotion recognition on average, and our presented SLSMKCCA and SSLSMKCCA fusion methods both obtain very high recognition rates, especially for the SSLSMKCCA fusion method.

Descriptive Characteristics of Systematic Functional Gestures Used by Pre-Service Earth Science Teachers in Classroom Learning Environments

Descriptive Characteristics of Systematic Functional Gestures Used by Pre-Service Earth Science Teachers in Classroom Learning Environments

Gesture recognition technology: a new dimension in human-computer interaction interface

Development of an interface for intelligent gesture control to improve user experience and increase the efficiency of interaction with a computer. This paper proposes a gesture recognition system based on artificial intelligence using convolutional neural networks (CNN). The system comprises three stages: pre-processing, optimal frame determination, and gesture category identification. The extracted features used are independent of movement, scaling, and rotation, providing greater flexibility to the system. The suggested gesture control technology, known as Kazakh Sign Language (KSL) for Kazakh alphabets, eliminates the need for additional devices, enabling users to interact with the system naturally. Experiments demonstrated that the proposed KSL system can accurately recognize Kazakh language alphabet letters with a high precision of 97.3%, owing to the utilization of artificial intelligence and CNN to enhance the accuracy and effectiveness of gesture control. Gestures, a type of visual formation, are perceivable by computers through machine learning models. The selection of methods and systems for recognizing Kazakh sign language gestures was accompanied by addressing various challenges related to language-specific orthographic and gestural features. The developed gesture control interface for human-computer interaction is applied in the field of inclusive education, aiming to assist deaf and hard-of-hearing children in learning sign language.

Learning hierarchical discrete prior for co-speech gesture generation

In the context of Co-Speech Gesture Generation, Vector-Quantized Variational Autoencoder (VQ-VAE) based methods have shown promising results by separating the generation process into two stages: learning discrete gesture priors via pretraining for gesture reconstruction, which encodes gesture into a discrete codebook, followed by learning the mapping between speech audio and gesture codebook indices. This design leverages pretraining of motion VQVAE with the motion reconstruction task to improve the quality of generated gestures. However, the vanilla VQVAE’s codebook often fails to encode both low-level and high-level gesture features adequately, resulting in limited reconstruction quality and generation performance. To address this, we propose the Hierarchical Discrete Audio-to-Gesture (HD-A2G), which innovates (i) a two-stage hierarchical codebook structure for capturing high-level and low-level gesture priors, enabling the reconstruction of gesture details. (ii) it further integrates high-level and low-level feature using an AdaIn layer, effectively enhancing the learning of gesture’s rhythm and content. (iii) it explicitly maps text and audio onset features to the appropriate levels of the codebook, ensuring learning accurate hierarchical associations for the generation stage. Experimental results on the BEAT and Trinity datasets demonstrate that HD-A2G outperform the baseline method in both pretrained gesture reconstruction and audio-conditioned gesture generation with a clear margin, achieving the state-of-the-art performance qualitatively and quantitatively.

Research on the Application of IU-EKF Algorithm for Feature Extraction of Flute Performers Based on Deep Learning

In order to achieve a more standardized teaching of flute playing gestures, a deep learning based application research on flute player hand shape feature extraction was proposed. Based on the theory of deep learning, a fusion network model based on VGG-16 was designed. The IU-EKF algorithm for deep learning was used to model, pose, learn, and train the hand shape of flute players. Experimental data analysis was conducted on the learning algorithm, and the experimental results showed that the model can extract gesture features for flute performance, whether on the back of the hand. The various angles of the upper and lower joints of the fingers have higher precision recognition than traditional methods. The feature extraction of flute gesture performance can also more effectively recognize the changes in different pitch angles of the gesture at different times, which has important reference value for the standardization of flute hand performance.

Study on Gesture Recognition Method with Two-Stream Residual Network Fusing sEMG Signals and Acceleration Signals.

Currently, surface EMG signals have a wide range of applications in human-computer interaction systems. However, selecting features for gesture recognition models based on traditional machine learning can be challenging and may not yield satisfactory results. Considering the strong nonlinear generalization ability of neural networks, this paper proposes a two-stream residual network model with an attention mechanism for gesture recognition. One branch processes surface EMG signals, while the other processes hand acceleration signals. Segmented networks are utilized to fully extract the physiological and kinematic features of the hand. To enhance the model's capacity to learn crucial information, we introduce an attention mechanism after global average pooling. This mechanism strengthens relevant features and weakens irrelevant ones. Finally, the deep features obtained from the two branches of learning are fused to further improve the accuracy of multi-gesture recognition. The experiments conducted on the NinaPro DB2 public dataset resulted in a recognition accuracy of 88.25% for 49 gestures. This demonstrates that our network model can effectively capture gesture features, enhancing accuracy and robustness across various gestures. This approach to multi-source information fusion is expected to provide more accurate and real-time commands for exoskeleton robots and myoelectric prosthetic control systems, thereby enhancing the user experience and the naturalness of robot operation.

Cognitive Classifier of Hand Gesture Images for Automated Sign Language Recognition: Soft Robot Assistance Based on Neutrosophic Markov Chain Paradigm

In recent years, Sign Language Recognition (SLR) has become an additional topic of discussion in the human–computer interface (HCI) field. The most significant difficulty confronting SLR recognition is finding algorithms that will scale effectively with a growing vocabulary size and a limited supply of training data for signer-independent applications. Due to its sensitivity to shape information, automated SLR based on hidden Markov models (HMMs) cannot characterize the confusing distributions of the observations in gesture features with sufficiently precise parameters. In order to simulate uncertainty in hypothesis spaces, many scholars provide an extension of the HMMs, utilizing higher-order fuzzy sets to generate interval-type-2 fuzzy HMMs. This expansion is helpful because it brings the uncertainty and fuzziness of conventional HMM mapping under control. The neutrosophic sets are used in this work to deal with indeterminacy in a practical SLR setting. Existing interval-type-2 fuzzy HMMs cannot consider uncertain information that includes indeterminacy. However, the neutrosophic hidden Markov model successfully identifies the best route between states when there is vagueness. This expansion is helpful because it brings the uncertainty and fuzziness of conventional HMM mapping under control. The neutrosophic three membership functions (truth, indeterminate, and falsity grades) provide more layers of autonomy for assessing HMM’s uncertainty. This approach could be helpful for an extensive vocabulary and hence seeks to solve the scalability issue. In addition, it may function independently of the signer, without needing data gloves or any other input devices. The experimental results demonstrate that the neutrosophic HMM is nearly as computationally difficult as the fuzzy HMM but has a similar performance and is more robust to gesture variations.

Product Design for the Elderly Based on Human-Computer Interaction in the Era of Big Data

Since 21st century, social aging trend has become more and more serious. It needs accurately identify the nursing needs of elderly disabled people, who are characterized by inconvenient movement and unclear speech. How to solve these problems has become a key point in the field of elderly care and medical care. In response to this problem, this research has designed a human-computer interactive gesture recognition system for elderly nursing beds in the context of big data. The recognition rate of the fusion feature + support vector machine (SVM) classifier adopted in this study is higher than 90% for each category of gesture. On the test set, this method has an average recognition rate of 96.35%, which is much higher than that of single feature + SVM classifier. While other methods’ recognition rate is lower than 90%. The recognition rate of tag c (nursing bed posture turning left) with obvious gesture feature information is as high as 99.28%, and that of tag h (nursing bed posture bedpan lowering) with weak gesture feature is 93.65%. The human-computer interaction system has well realized the recognition intention of user’s dynamic and static gestures, achieved the goal set by the research, and the interaction form is natural and reliable. In the later research, we can further realize a more comprehensive, accurate and natural human-computer interaction product design through the multi-channel joint decision-making scheme to meet the needs of the elderly.

Musculoskeletal disorders in padel: from biomechanics to sonography

Padel is a racket sport, combining high-frequency and low-intensity athletic gestures, that has been gaining growing scientific interest in recent years. Musculoskeletal injuries are very common among padel players with an incidence rate of 3 per 1000 h of training and 8 per 1000 matches. To the best of our knowledge, a comprehensive collection describing the most common sonographic findings in padel players with musculoskeletal injuries is lacking in the pertinent literature. In this sense, starting from the biomechanical features of padel-specific gestures we have reported the ultrasonographic patterns of most frequent injuries involving the upper limb, the trunk, and the lower limb. Indeed, comprehensive knowledge of the biomechanical and clinical features of musculoskeletal injuries in padel is paramount to accurately perform a detailed ultrasound examination of the affected anatomical site. So, the present investigation aims to provide a practical guide, simple and ready-to-use in daily practice, to optimize the sonographic assessment of padel players by combining it with the clinical findings and the biomechanical features of athletic gestures.

Developmental organization of the graphic gesture with a pre-scruptural task to assess handwriting

IntroductionThe literature mainly focused on spatio-temporal and kinematics parameters of tracing letters or words using digitizing tablets, no recent research has previously studied the developmental prerequisites of the organization of handwriting.ObjectivesWe aimed to aimed to investigate and validate the developmental organization of the graphic gesture with a pre-scruptural task.Methods122 typically developing right-handed elementary school children (grades 1st to 5th) aged from 6 to 11;3 years old were recruited. The axe postural and arm gestural features were video-recorded with analysis in 2D reconstruction. Spatial (length, size, regularity, slope of the line…), temporal (drawing time, pause time) and kinematic measures (velocity, peak velocity) were collected with a digital pen independent connected to an analysis software tool. External validity was studied in relation with the standardized handwriting scale BHK. The child has to draw a line of cycloid loops (from left to right drawn in an anti-clockwise direction) across the width of an A4 size unlined half sheet of white paper (containing non-visible watermarks to provide the location of the pen) free to move on the table, after observing the dynamic model on the iPad placed in front of him.ResultsFive main patterns of inter-segmental displacement gestures were found for the production of the line of loops with a significant developmental progress from grades 1st to 5th. Findings showed significant economic rotation movement with forearm rotation around the elbow in 4th and 5th grade, with the elbow tending significantly to be static on the table (p = 2.43e-16), wrist on the table and (p =0.02) and in half-supine position (p =0·001), tri-digital grasp of the pen (p =3.81e-08). Moreover, the mean pressure applied on the pen decrease at 4 and 5th grades and it is correlated to deleterious spatial-temporal and kinematic parameters.ConclusionsThe results of our study provide the first developmental grade and age-related normative data in the developmental genesis of the graphomotor gesture and with the spatio-temporal /kinematic measures. The more mature the gesture, the more there is a decrease in degrees of freedom of movement and stabilization of the joints that is fixed, as well as the presence of a distal flexion movement extending fingers in synergy with the rotation of the arm around the elbow. Furthermore, the task of copying of loops in ecological settings appears to be a good predictor for legibility and writing speed. Those data can account for the mechanisms of motor programming necessary to the automatization of the future gesture of handwriting.Disclosure of InterestNone Declared

Next-Generation Captcha: Enhancing Security Through Hand Gestures and Gaming

Abstract: In the digital age, the threat of automated attacks on online platforms continues to evolve, necessitating innovative approaches to ensure cybersecurity. Traditional text-based CAPTCHA systems are becoming increasingly vulnerable to sophisticated attacks, prompting the exploration of alternative solutions. This research proposes a novel approach to enhance online security through the integration of hand gestures and gaming elements into CAPTCHA mechanisms. By leveraging the unique biometric characteristics of hand gestures and the engaging nature of gaming, this next-generation CAPTCHA system aims to provide robust protection against automated bots while ensuring a user-friendly experience. The study explores the design, implementation, and evaluation of this hybrid CAPTCHA solution, assessing its effectiveness in mitigating various challenges associated with existing CAPTCHA methods. Through empirical analysis and user feedback, the research aims to demonstrate the feasibility and efficacy of incorporating hand gestures and gaming into CAPTCHA systems to enhance security and usability in online environments.

Robust gesture recognition method toward intelligent environment using Wi-Fi signals

Wi-Fi-based gesture recognition represents an emerging paradigm of human–computer interaction. While deep learning-based and model-based solutions are crucial for improving accuracy and generalization performance, they often depend heavily on extensive datasets and the widespread deployment of sensing devices. In this paper, we address the challenge involving limited sensing data and devices. We propose a signal-based gesture recognition method that leverages gesture coding based on stroke trajectory and feature representation based on dynamic phase changes. Specifically, we develop an endpoint detection algorithm to ensure precise gesture recognition. Additionally, a subcarrier selection algorithm is designed to select optimal subcarriers, capturing comprehensive gesture information. Extensive experiments are conducted to evaluate the performance. Results demonstrate an average accuracy of 93.67% for six gestures. This approach effectively mitigates the impact of location and environment factors on gesture characteristics, and reduces dependence on large quantity of samples and transceiver devices, obviating the need for model training.

Preboundary lengthening and articulatory strengthening in Korean as an edge-prominence language

This study examined preboundary lengthening and other kinematic characteristics of articulatory gestures in CV.CV and CV.CVC before prosodic boundaries in Korean. Preboundary lengthening was found to be extended to initial syllables in both CV.CVand CV.CVC, while its magnitude was largest on the final syllable. The preboundary lengthening effect was also reflected in the time-to-peak velocity (acceleration duration), but only on gestures of the final syllable. Preboundary lengthening was accompanied by substantial increase in both displacement and peak velocity, showing domain-final articulatory strengthening. This articulatory strengthening effect on preboundary gestures (at the right edge of prosodic constituent) was largely dovetailed with the notion of an edge-prominence language where boundary marking is assumed to be closely related with prominence lending. These results were compared in two different conditions driven by information structure (‘new’ vs. ‘given’) and were discussed to understand the observed kinematic pattern in dynamical terms in the theoretical framework of the π-gesture model. 

Decoding surgical skill: an objective and efficient algorithm for surgical skill classification based on surgical gesture features -experimental studies.

Various surgical skills lead to differences in patient outcomes and identifying poorly skilled surgeons with constructive feedback contributes to surgical quality improvement. The aim of the study was to develop an algorithm for evaluating surgical skills in laparoscopic cholecystectomy based on the features of elementary functional surgical gestures (Surgestures). Seventy-five laparoscopic cholecystectomy videos were collected from 33 surgeons in five hospitals. The phase of mobilization hepatocystic triangle and gallbladder dissection from the liver bed of each video were annotated with 14 Surgestures. The videos were grouped into competent and incompetent based on the quantiles of modified global operative assessment of laparoscopic skills (mGOALS). Surgeon-related information, clinical data, and intraoperative events were analyzed. Sixty-three Surgesture features were extracted to develop the surgical skill classification algorithm. The area under the receiver operating characteristic curve of the classification and the top features were evaluated. Correlation analysis revealed that most perioperative factors had no significant correlation with mGOALS scores. The incompetent group has a higher probability of cholecystic vascular injury compared to the competent group (30.8 vs 6.1%, P =0.004). The competent group demonstrated fewer inefficient Surgestures, lower shift frequency, and a larger dissection-exposure ratio of Surgestures during the procedure. The area under the receiver operating characteristic curve of the classification algorithm achieved 0.866. Different Surgesture features contributed variably to overall performance and specific skill items. The computer algorithm accurately classified surgeons with different skill levels using objective Surgesture features, adding insight into designing automatic laparoscopic surgical skill assessment tools with technical feedback.

Frequency and Spatial Domain-Based Approaches for Recognition of Indian Sign Language Gestures

Objectives: The objective of this paper is to introduce and demonstrate an innovative approach for the recognition of Indian sign language gestures, with a focus on bridging communication gap between the deaf and hearing communities. The goal is to contribute to the development of effective tools and technologies that facilitate seamless communication between individuals using sign language and the people with no knowledge about sign language. Methods: The methodology consists of three key steps. First, data pre-processing involves resizing and contours extraction. Next, feature extraction employs Fourier descriptors for frequency domain analysis and gray-level-co-occurrence matrix for spatial domain analysis. Finally, various machine learning models including SVM, Random Forest, Logistic Regression, K-Nearest Neighbor and Naive Bayes are trained on a standard dataset. Findings: In our controlled experimental setup, we applied a diverse set of machine learning classifiers to evaluate the proposed approach for gesture recognition. Among the classifiers tested, K-Nearest Neighbors demonstrated the highest accuracy, achieving 99.82%. To validate the robustness of our approach, we employed k-fold cross-validation with 5 folds. Novelty: This study presents an innovative method for sign language recognition by employing a dual-domain fusion strategy that prominently emphasizes the frequency domain. Through the integration of Fourier descriptors, the research conducts a detailed frequency domain analysis to characterize the contour shapes of sign language gestures. The synergy with gray-level co-occurrence matrix texture features in the spatial domain analysis, contributes to the creation of a comprehensive feature vector. The proposed approach ensures a thorough exploration of gesture features, there by advancing the precision and efficacy of sign language recognition. Keywords: Indian Sign Language (ISL), Sign Language Recognition (SLR), Frequency domain, Spatial domain, Fourier descriptors, Gray level co­occurrence matrix (GLCM), K­ Fold

Research on Informatization Construction and Teaching Reform Path of Physical Education Teaching in Colleges and Universities

Abstract Firstly, the improved OpenPose is used to obtain the skeletal point coordinate data and normalized, and then the human gesture features are extracted according to the spatial geometric relationship of human morphology to improve the differentiation degree of the gesture features on similar actions in the basic sports actions, which is used for the gesture matching of the basic sports actions. On this basis, the DTW posture matching algorithm based on human differentiated posture features is proposed, which realizes the posture matching and carries out the evaluation and result feedback of students’ sports movements. Finally, we construct a sports movement evaluation system based on the aforementioned method, which compares the movement video with the standard movement from the template library and provides feedback on the movement’s evaluation results. The system allows teachers to access the learning situations of numerous students’ movements, while the system provides feedback for movement learners to adjust their movements, thereby facilitating a convenient and informative interaction in sports movement teaching. After seven weeks of experimentation, there is a significant difference between the assessment scores of the experimental group and the control group, and 90% of the students are satisfied with the application of the sports movement assessment teaching system to assist teaching. It proves that the intelligent teaching structure and diversified teaching methods promote students’ learning efficiency and performance, significantly improve students’ motivation to learn physical education courses, and further improve the demand for an information-based teaching environment.