Types Of Gestures Research Articles

Creative Action at a Distance: A Conceptual Framework for Embodied Performance With Robotic Actors.

Acting, stand-up and dancing are creative, embodied performances that nonetheless follow a script. Unless experimental or improvised, the performers draw their movements from much the same stock of embodied schemas. A slavish following of the script leaves no room for creativity, but active interpretation of the script does. It is the choices one makes, of words and actions, that make a performance creative. In this theory and hypothesis article, we present a framework for performance and interpretation within robotic storytelling. The performance framework is built upon movement theory, and defines a taxonomy of basic schematic movements and the most important gesture types. For the interpretation framework, we hypothesise that emotionally-grounded choices can inform acts of metaphor and blending, to elevate a scripted performance into a creative one. Theory and hypothesis are each grounded in empirical research, and aim to provide resources for other robotic studies of the creative use of movement and gestures.

Intra-Brain Connectivity vs. Inter-Brain Connectivity in Gestures Reproduction: What Relationship?

Recently, the neurosciences have become interested in the investigation of neural responses associated with the use of gestures. This study focuses on the relationship between the intra-brain and inter-brain connectivity mechanisms underlying the execution of different categories of gestures (positive and negative affective, social, and informative) characterizing non-verbal interactions between thirteen couples of subjects, each composed of an encoder and a decoder. The study results underline a similar modulation of intra- and inter-brain connectivity for alpha, delta, and theta frequency bands in specific areas (frontal or posterior regions) depending on the type of gesture. Moreover, taking into account the gestures’ valence (positive or negative), a similar modulation of intra- and inter-brain connectivity in the left and right sides was observed. This study showed congruence in the intra-brain and inter-brain connectivity trend during the execution of different gestures, underlining how non-verbal exchanges might be characterized by intra-brain phase alignment and implicit mechanisms of mirroring and synchronization between the two individuals involved in the social exchange.

Augmenting Around-Device Interaction by Geomagnetic Field Built-in Sensor Utilization.

In this paper, we investigate the possibilities for augmenting interaction around the mobile device, with the aim of enabling input techniques that do not rely on typical touch-based gestures. The presented research focuses on utilizing a built-in magnetic field sensor, whose readouts are intentionally affected by moving a strong permanent magnet around a smartphone device. Different approaches for supporting magnet-based Around-Device Interaction are applied, including magnetic field fingerprinting, curve-fitting modeling, and machine learning. We implemented the corresponding proof-of-concept applications that incorporate magnet-based interaction. Namely, text entry is achieved by discrete positioning of the magnet within a keyboard mockup, and free-move pointing is enabled by monitoring the magnet’s continuous movement in real-time. The related solutions successfully expand both the interaction language and the interaction space in front of the device without altering its hardware or involving sophisticated peripherals. A controlled experiment was conducted to evaluate the provided text entry method initially. The obtained results were promising (text entry speed of nine words per minute) and served as a motivation for implementing new interaction modalities. The use of neural networks has shown to be a better approach than curve fitting to support free-move pointing. We demonstrate how neural networks with a very small number of input parameters can be used to provide highly usable pointing with an acceptable level of error (mean absolute error of 3 mm for pointer position on the smartphone display).

Does a pedagogical agent\u2019s gesture frequency assist advanced foreign language users with learning declarative knowledge?

Since the conception of pedagogical agents in multimedia environments, researchers have advocated for agents to be designed to exhibit social cues that prime the social interaction of the target audience. One powerful social cue has been agent gesturing. While most agents are created only to use deictic (pointing) gestures, there is recent evidence that agents that perform all gesture types (iconic, metaphoric, deictic, and beat) with enhanced frequency help foreign language users learn more procedural knowledge. Therefore, this research examines how all gesture types and different frequencies influence agent persona and learning outcomes when foreign language users learn declarative knowledge. The results indicated that the use of gestures, regardless of frequency, significantly increase agent persona. However, gesture frequency produced conflicting learning outcomes. While enhancing gestures were beneficial for cued recall and recognition, the average gesture condition was not, which indicates that the strength of social cues is important.

Hand Gesture Recognition By Deep Convolutional Neural Network

Today the hand gesture is increasingly in the human-computer interactions. In all project, there is a minimum number of the process involved in the hands only and maximum contains the body, background, etc., In this paper, it shows that the location of the hand, hand gesture, and applies it in home and also the interaction between the system and the users. Hand gesture means that it is the form of sign language in the form of passing the signals and expressions. The movement occurred in the hand is called the hand gesture which can be used in hearing loss peoples. The processing of the hand gesture is that it shares the communication to others with the help of hands. The different symbols and numbers are forming with the hands and the signs form a group of words that can be understood by other people. There are two types of hand gestures like a glove based and vision-based.In this paper, a new approach called deep convolutional neural networks, which used in hand gestures to classify the images held in the hand gesture. The deep convolutional hand gesture can be used to find a more number of hand gestures. The hand gestures require more time for processing. The convolutional neural network can be used for saving the timing and reducing the time used in the separation of things or images. Finding the state of hand gesture, which will also, needs more time. The deep CNN provides a faster process in that. Using the deep CNN method, provide the system with more accuracy, specificity, sensitivity compared to the other systems. The proposed method as CNN, which separates the different sections, provides good accuracy, specificity, sensitivity from the dataset. The system process ends until the hand gesture image is leaving from the camera.

Dig That Lick (DTL): Analyzing Large-Scale Data for Melodic Patterns in Jazz Performances

Dig That Lick (DTL): Analyzing Large-Scale Data for Melodic Patterns in Jazz Performances

A Scalable Solution for Signaling Face Touches to Reduce the Spread of Surface-based Pathogens

Hand-to-Face transmission has been estimated to be a minority, yet non-negligible, vector of COVID-19 transmission and a major vector for multiple other pathogens. At the same time, as it cannot be effectively addressed with mainstream protection measures, such as wearing masks or tracing contacts, it remains largely untackled. To help address this issue, we have developed Saving Face - an app that alerts users when they are about to touch their faces, by analyzing the distortion patterns in the ultrasound signal emitted by their earphones. The system only relies on pre-existing hardware (a smartphone with generic earphones), which allows it to be rapidly scalable to billions of smartphone users worldwide. This paper describes the design, implementation and evaluation of the system, as well as the results of a user study testing the solution's accuracy, robustness, and user experience during various day-to-day activities (93.7% Sensitivity and 91.5% Precision, N=10). While this paper focuses on the system's application to detecting hand-to-face gestures, the technique can also be applicable to other types of gestures and gesture-based applications.

Surgical Hand Gesture Recognition Utilizing Electroencephalogram as Input to the Machine Learning and Network Neuroscience Algorithms.

Surgical gestures detection can provide targeted, automated surgical skill assessment and feedback during surgical training for robot-assisted surgery (RAS). Several sources including surgical videos, robot tool kinematics, and an electromyogram (EMG) have been proposed to reach this goal. We aimed to extract features from electroencephalogram (EEG) data and use them in machine learning algorithms to classify robot-assisted surgical gestures. EEG was collected from five RAS surgeons with varying experience while performing 34 robot-assisted radical prostatectomies over the course of three years. Eight dominant hand and six non-dominant hand gesture types were extracted and synchronized with associated EEG data. Network neuroscience algorithms were utilized to extract functional brain network and power spectral density features. Sixty extracted features were used as input to machine learning algorithms to classify gesture types. The analysis of variance (ANOVA) F-value statistical method was used for feature selection and 10-fold cross-validation was used to validate the proposed method. The proposed feature set used in the extra trees (ET) algorithm classified eight gesture types performed by the dominant hand of five RAS surgeons with an accuracy of 90%, precision: 90%, sensitivity: 88%, and also classified six gesture types performed by the non-dominant hand with an accuracy of 93%, precision: 94%, sensitivity: 94%.

WiFi CSI gesture recognition based on parallel LSTM-FCN deep space-time neural network

In this study, we developed a system based on deep space-time neural networks for gesture recognition. When users change or the number of gesture categories increases, the accuracy of gesture recognition decreases considerably because most gesture recognition systems cannot accommodate both user differentiation and gesture diversity. To overcome the limitations of existing methods, we designed a one- dimensional parallel long short-term memory-fully convolutional network (LSTM-FCN) model to extract gesture features of different dimensions. LSTM can learn complex time dynamic information, whereas FCN can predict gestures efficiently by extracting the deep, abstract features of gestures in the spatial dimension. In the experiment, 50 types of gestures of five users were collected and evaluated. The experimental results demonstrate the effectiveness of this system and robustness to various gestures and individual changes. Statistical analysis of the recognition results indicated that an average accuracy of approximately 98.9% was achieved.

The autocracy of meaning: Intact visuo-imitative processes may not compensate for meaningful gestures

The dual-route models of action distinguish between a semantic and a non-semantic visuo-motor route to execute different types of gestures. Despite the large amount of evidence in support to the model, some aspects are debated. One issue concerns the recruitment of the visuo-motor route to correctly execute meaningful gestures when the semantic route is damaged. Debated predictions of the dual-route model were investigated in a sample of 32 patients with left hemisphere stroke lesions compared to 27 healthy controls. Group analysis showed that patients were equally impaired on meaningful and meaningless gestures. Single-case analysis demonstrated that most cases were more impaired on meaningful than on meaningless gestures both when they are given in separate lists and when they are intermingled. Impaired performance on the imitation of meaningful gestures in both the separate and mixed list but spared performance on meaningless gestures in the separate list is against the hypothesis that the intact visuo-motor route compensates for damage to the semantic route. These results suggest that the damaged semantic route interferes with the visuo-motor route and prevents the processing of meaningful gestures along the visuo-motor route. Furthermore, an explorative analysis was conducted to investigate the relationship between gestures imitation and pantomime of object use on verbal command and between gestures imitation and performance on linguistic tasks. Although no significant correlation emerged, patients with moderate/severe impairment on the AAT performed significantly worse on meaningful, but not on meaningless gestures than patients with mild/minimal language impairment, suggesting that praxis and language systems are independent but dynamically interact.

Dynamic Gesture Recognition System with Gesture Spotting Based on Self-Organizing Maps

In this paper, a real-time dynamic hand gesture recognition system with gesture spotting function is proposed. In the proposed system, input video frames are converted to feature vectors, and they are used to form a posture sequence vector that represents the input gesture. Then, gesture identification and gesture spotting are carried out in the self-organizing map (SOM)-Hebb classifier. The gesture spotting function detects the end of the gesture by using the vector distance between the posture sequence vector and the winner neuron’s weight vector. The proposed gesture recognition method was tested by simulation and real-time gesture recognition experiment. Results revealed that the system could recognize nine types of gesture with an accuracy of 96.6%, and it successfully outputted the recognition result at the end of gesture using the spotting result.

Proposing a Recognition System of Gestures Using MobilenetV2 Combining Single Shot Detector Network for Smart-Home Applications

The paper proposes a system for identifying gestures and actions in smart homes. The proposed method is based on MobilenetV2 feature extraction combining with single shot detector (SSD) network. We used eleven types of gestures of walking, sitting down, falling back, wearing shoes, waving hands, falling down, smoking, baby crawling, standing up, reading, and typing for recognizing the gestures. In this system, the data are captured from the camera of mobile devices that are used to detect the object. The results are obtained objects on the frame by a bounding box. The results show that the system meets the requirements with an accuracy of over 90% that is suitable for real application.

Frequency of Gesture Use and Language in Typically Developing Prelinguistic Children

PurposeThe aim of this study was to examine the relationship between frequency of gesture use and language with a consideration for the effect of age and setting on frequency of gesture use in prelinguistic typically developing children. MethodParticipants included a total of 54 typically developing infants and toddlers between the ages of 9 months and 15 months separated into two age ranges, 9-12 months and 12-15 months. All participants were administered the Mullen’s Scale of Early Learning and two gesture samples were obtained: one sample in a structured setting and the other in an unstructured setting. Gesture samples were coded by research assistants blind to the purpose of the research study and total frequency and frequencies for the following gesture types were calculated: behavior regulation, social interaction, and joint attention (Bruner, 1983). ResultsResults indicated that both age and setting have a significant effect on frequency of gesture use and frequency of gesture is correlated to receptive and expressive language abilities; however, these relationships are dependent upon the gesture type examined. ConclusionsThese findings further our understanding of the relationship between gesture use and language and support the concept that frequency of gesture is related to language abilities. This is meaningful because gestures are one of the first forms of intentional communication, allowing for early identification of language abilities at a young age.

BRICS – quo vadis?

ABSTRACT The BRICS countries gained importance on the international stage with two opposing scenarios from the literature dictating discussions. The first proclaimed that BRICS would become a major economic force globally, while the second perceived it as a mere symbolic type of gesture with minimal impact. Based on this, should the group be able to fully converge and unite, the first may become true, but if the second narrative becomes reality, the group may diverge with limited impact globally. To become an economic entity that has significant influence in international affairs, unity rather than diversity is imperative. Following a desk-top literature approach in this study, it seems to indicate that BRICS forms a heterogeneous coalition, focussed on advancing members’ individual, rather than collective interest, using the benefit of being aligned with a potential economic powerhouse. Therefore, it seems as if the second scenario, seems more plausible.

Gesture, prosody and verbal content in non-fluent aphasic speech

Abstract Non-fluent aphasia is characterized by frequent word search and a much slower speech rate than non-aphasic speech. For patients with this type of aphasia, communication with those around them is therefore made difficult and is often severely impaired. One of the therapeutic proposals to improve the quality of life of these patients is to re-educate them with more multimodal alternatives. This of course assumes that gestures represent possible alternative means of communication for patients, and that their gestures are not affected in the same way as their speech. This article therefore proposes to study the gestures of 4 aphasic people and to compare them to the gestures performed by non-aphasic people, but also to establish correspondences between those gestures, intonation contours and the way people with aphasia develop their discourse. Results show that although gesture rate is not different in the two groups of participants, the gesture-to-speech ratio is higher for people with aphasia (PWA) than for non-aphasic people (NAP). Considering the fact that PWA also gesture more than NAP during silent pauses, which are longer but not more frequent than in NAP’s speech, and the fact that their gestures coincide less often with a lexical word, we believe that PWA use their gestures as compensation strategies for deficient speech. Yet, their speech impairment is also reflected in their gesturing: more gestures are prepared but abandoned before the stroke in this group and pre-stroke holds are longer, which means that PWA hold their gestures in the hope that they will better coincide with the word they are supposed to accompany and which takes more time to be uttered than in non-pathological speech. Their gestures are also less linked to each other than in the NAP group which goes hand in hand with the fact that they tend to utter independent syntactic phrases with no cohesive marker between them. This is also reflected in their less frequent use of flat and rising tones in intonation, which generally indicate that two sentence parts are dependent one upon the other, as well as their less frequent use of gestures showing discourse organization. In terms of gesture types, the PWA in this study perform many rhythmic beats and rely much on conventional gestures to compensate for their speech impairment rather than on their own creativity. Globally, this means that if multimodal therapies may benefit PWA to improve their communication with other people, speech therapists nevertheless need to be aware that life-long habits of gesture-speech alignment and synchronization may not be so easy to overcome for patients.

Multimodal Russian Corpus and its use in emotional studies

ABSTRACT The paper introduces the Multimodal Russian Corpus (MURCO) as a part of the Russian National Corpus (RNC). MURCO provides users with a great amount of linguistic information related to Russian. Moreover, the deeply annotated part of the MURCO contains data concerning Russian gesticulation, speech act system, types of vocal gestures and interjections in Russian, and so on. It should be noted that there is no specific annotation of emotions in MURCO, because it was initially designed as a tool for the linguistic research of different types of oral discourse. However, using various types of annotation, and relying on the correlation between emotions and corresponding language features, we can find a great number of emotional contexts for further study. This article describes the main types of annotation and the structure of the MURCO interface; and demonstrates various ways to extract the necessary information for emotional studies.

САМОИСПРАВЛЕНИЯ В ПРЕРВАННЫХ ВЫСКАЗЫВАНИЯХ: ПОЛИМОДАЛЬНЫЙ АНАЛИЗ СИНХРОННОГО ПЕРЕВОДА

The article presents the results of an empirical research dedicated to the co-occurrence of gestures and self-repairs in simultaneous interpreting. Self-repairs, viewed as a type of disfluencies, are divided into three categories: 1) a disfluency followed by a resolution (“positive” self-repair), 2) a disfluency consisting of repetition of the same lexical unit (“zero” self-repair); 3) an utterance truncated without a restart, which means that such a disfluency is not resolved. The study is based on the assumption that in simultaneous interpreting statistically significant correlation will be observed between various types of self-repair and various types of gestures, such as pragmatic, representational, deictic gestures and adapters. The data was obtained from 18 interpreters who were asked to interpret a popular science lecture from Russian into English. The material was analysed in ELAN and JAMOVI, and quantitative and statistical methods (T-test and ANOVA) were used to check on the hypotheses. The study revealed a strong correlation between self-repairs and gesture usage, which suggests that in moments of disfluency gestures facilitate speech. Concerning the relation between various types of self-repair and functional types of gestures, the hypothesis was not confirmed, although the statistics points to the fact that the simultaneous interpreters tend to use particular types of gestures with certain self-repairs.

Retracted] Using Depth Cameras for Recognition and Segmentation of Hand Gestures

In recent years, in combination with technological advances, new paradigms of interaction with the user have emerged. This has motivated the industry to create increasingly powerful and accessible natural user interface devices. In particular, depth cameras have achieved high levels of user adoption. These devices include the Microsoft Kinect, the Intel RealSense, and the Leap Motion Controller. This type of device facilitates the acquisition of data in human activity recognition. Hand gestures can be static or dynamic, depending on whether they present movement in the image sequences. Hand gesture recognition enables human‐computer interaction (HCI) system developers to create more immersive, natural, and intuitive experiences and interactions. However, this task is not easy. That is why, in the academy, this problem has been addressed using machine learning techniques. The experiments carried out have shown very encouraging results indicating that the choice of this type of architecture allows obtaining an excellent efficiency of parameters and prediction times. It should be noted that the tests are carried out on a set of relevant data from the area. Based on this, the performance of this proposal is analysed about different scenarios such as lighting variation or camera movement, different types of gestures, and sensitivity or bias by people, among others. In this article, we will look at how infrared camera images can be used to segment, classify, and recognise one‐handed gestures in a variety of lighting conditions. A standard webcam was modified, and an infrared filter was added to the lens to create the infrared camera. The scene was illuminated by additional infrared LED structures, allowing it to be used in various lighting conditions.

An Indian Corpus of Gestures for Nouns and Verbs: A Preliminary Study

The study aimed to develop a corpus of comprehensive gestures for a set of nouns and verbs in the Indian context. The study was conducted in the following three phases. Phase 1: Gestural nouns and verbs were selected from the existing word lists. Phase 2: A set of 221 nouns and 122 verbs were enacted and recorded by a professional actor and a cameraman respectively. Phase 3: Two groups of participants i.e., 15 young (18-39 years) and 15 old (40-69 years) neuro-typical adults participated in the two tasks namely (a) naming and (b) rating tasks. The analyses of the responses revealed maximum uniformity of correct responses between the participants at 85% agreement level for both noun and verb gestures. Thus, a set of 86 noun and 60 verb gesture videos entered the final gesture corpus which satisfied the criteria. Further, the varied responses among the participants in interpreting the gestures and type of gestures have been discussed and documented. The corpus developed can serve as stimulus in various gesture studies. Although the corpus comes with few limitations, the data provided would help the researchers to carefully select the gesture videos based on their purpose of the study.

Surface-Electromyography-Based Gesture Recognition Using a Multistream Fusion Strategy

Gestures are an important way to conduct human-computer interaction. The key problem of gesture recognition depending on sEMG (surface electromyography) is how to achieve high recognition accuracy when there are many types of gestures to classify. To solve this problem, first, two basic models were constructed. One is the ConvEMG model based on dense connectivity, the Inception module and depthwise separable convolution; and the other is the LSTMEMG model based on a bidirectional LSTM (Long Short-Term Memory). Then, the basic models were improved with a multistream fusion strategy which utilizes the correlation between gestures and muscles and the complementary advantages of models. To facilitate comparison with others’ models, the models proposed in this paper were tested on the public dataset NinaPro DB5, and the improved model named MultiConvEMG achieves an accuracy of 92.83% for 41 gestures, which is superior to its counterparts in the literature on the same dataset. In addition, experiments containing signal acquisition and gesture recognition were carried out for further testing and evaluation. Experimental results show that all models can achieve an accuracy of more than 95% for 31 gestures, and these models have their own strengths in accuracy, immediacy or training cost. All models built in the paper support using sEMG for end-to-end recognition, which means that artificial features are not needed in the processes and data augmentation or IMU devices are not relied on. In other words, our models outperform and have lower application costs than many known models.