Types Of Gestures Research Articles

Gestures, Objects, and Spaces: Exploring Teachers' Multimodal Communication in Nursery Schools.

This study builds on the increasing evidence that the multimodal nature of adult-child interactions and the use of objects play an important role in early linguistic development. Most of these studies analyzed dyadic interactions at home, whereas few research has been conducted in early childhood education and care settings. In this paper, we characterized the multimodal nature of teachers' communicative bids during classroom-based group interactions in nursery schools. Observational data of circle-time activities was collected from 16 Spanish nursery school classrooms, comprising 16 teachers and 161 children between two and three years of age. We analyzed teachers' communicative bids (i.e., verbal utterances and verbal-gestural bids) considering the frequency of use of different types of gestures, to whom are they addressed (i.e., the whole group or a single child), the extent to which they involve the use of objects, the classroom layout, and the relationship between the communicative bids and the number of children that participated in each classroom. Teachers' communication with toddlers is highly multimodal and rely on different types of gestures, although the use of objects in our sample was scarce. Descriptive analysis suggest that certain classroom layouts may favor teachers' use of some types of gestures over others. In this article, we discuss the implications of both the use of objects and space for understanding how adults shape the linguistic contexts of young children, and the potential opportunities and limitations they pose for classroom interactions.

Can Plants Perceive Human Gestures? Using AI to Track Eurythmic Human-Plant Interaction.

This paper explores if plants are capable of responding to human movement by changes in their electrical signals. Toward that goal, we conducted a series of experiments, where humans over a period of 6 months were performing different types of eurythmic gestures in the proximity of garden plants, namely salad, basil, and tomatoes. To measure plant perception, we used the plant SpikerBox, which is a device that measures changes in the voltage differentials of plants between roots and leaves. Using machine learning, we found that the voltage differentials over time of the plant predict if (a) eurythmy has been performed, and (b) which kind of eurythmy gestures has been performed. We also find that the signals are different based on the species of the plant. In other words, the perception of a salad, tomato, or basil might differ just as perception of different species of animals differ. This opens new ways of studying plant ecosystems while also paving the way to use plants as biosensors for analyzing human movement.

Gestural Alignment in Spoken Simultaneous Interpreting: A Mixed-Methods Approach

Cognitive and behavioral alignment plays a major role in simultaneous interpreting, the interpreter centrally monitoring and accommodating his/her behavior to that of the speaker-source. In parallel, the place of gesture in the interpreters’ practice, as well as its degree of convergence with respect to the gestures of the speaker-source, has been scarcely analyzed until very recently. The multimodal data for this study were collected under (quasi-)experimental conditions in a real court interpreting setting during spoken training exercises performed by two novice interpreters. In this exploratory study, the gestural performance of the interpreters, including their degree of gestural alignment towards the speaker-source, is analyzed and compared using a mixed-methods approach to a randomized sample of the recorded data. The analysis combines a basic descriptive quantification of body movements and a qualitative and comparative analysis of the gesture types performed by the speaker-source and the interpreters. The results show that, in spite of individual differences in interpreting fluency and gestural styles, both interpreters tend to align with the speaker-source’s gestural behavior in several ways, and thus a basic taxonomy of gestural convergence between the speaker-source and the interpreters is defined according to several criteria (mainly, gesture presence and gesture type). Our conclusions also allow us to formulate new research questions and hypotheses to be tested in future studies (e.g., types of gestures by the speaker-source that prompt a higher degree of alignment).

Egocentric and Allocentric Spatial Memory for Body Parts: A Virtual Reality Study.

Extensive literature elucidated the mechanisms underlying the ability to memorize the positions of objects in space. However, less is known about the impact that objects' features have on spatial memory. The present study aims to investigate differences in egocentric and allocentric object-location memory between hand stimuli depicted in a first-person perspective (1PP) or in a third-person one (3PP). Fifty-two adults encoded spatial positions within a virtual museum environment featuring four square buildings. Each of these buildings featured eight paintings positioned along the walls, with two pictures displayed on each of the four walls. Thirty-two stimuli were employed, which represented pictures of the right hand performing various types of gestures. Half of the stimuli depicted the hand in the 1PP, while the other half depicted the hand in the 3PP. Both free and guided explorations served as encoding conditions. Immediately after that, participants underwent a two-step object-location memory task. Participants were provided with a map of the museum and asked to identify the correct building where the image was located (allocentric memory). Then, they were presented with a schematic representation of the exhibition room divided into four sections and instructed to select the section where they thought the picture was located (egocentric memory). Our findings indicate a memory performance boost associated with egocentric recall, regardless of the perspective of the bodily stimuli. The results are discussed considering the emerging literature on the mnemonic properties of body-related stimuli for spatial memory.

Task-related differences in the gesture production of young autistic children.

In typically developing (TD) children, gesture emerges around 9months of age, allowing children to communicate prior to speech. Due to the important role gesture plays in the early communication of autistic and TD children, various tasks have been used to assess gesture ability. However, few data exist on whether and how tasks differentially elicit gesture, particularly for samples of racially and ethnically diverse autistic children. In this study, we explored if task (a naturalistic parent-child interaction [NPCI]; structured assessment of child communication) differentially elicited rate or type of gesture production for young autistic children. This secondary analysis included baseline data from 80 racially and ethnically diverse autistic children aged 18-59months who participated in one of two larger studies. Video recordings of NPCIs and an assessment of child communication with standardised administration procedures were collected at baseline. Child gesture rate (number of gestures produced per 10min) and type were extracted from these recordings and analysed. The structured assessment elicited more gestures than the NPCI. In terms of gesture type, points, gives, and reaches accounted for 76% of child gestures. Points (which are developmentally more advanced than reaches and gives) were produced at the highest rates within book exploration. Distal points (which are more developmentally advanced than proximal or contact points) were produced at the highest rates when children were tempted to request. Our findings indicate elicitation tasks differentially elicit type and rate of gesture for young autistic children. To assess the gesture production of young autistic children, a structured task designed to elicit child requests will probe the developmental sophistication of the child's gesture repertoire, eliciting both the most gestures and the most developmentally advanced gestures. What is already known on the subject Because of the importance of gesture in early communication for autistic and typically developing children, various tasks have been used to assess it. However, little is known about whether tasks differentially elicit type or rate of gesture for young autistic children from diverse racial and ethnic backgrounds. What this paper adds to existing knowledge Elicitation tasks differentially elicit type and rate of gesture for young autistic children in the early stages of gesture. What are the potential or actual clinical implications of this work? We recommend a structured task designed to elicit child requests to assess the developmental sophistication of a child's gesture repertoire.

Human motion activity recognition and pattern analysis using compressed deep neural networks

ABSTRACT This work presents an on-device machine learning model with the ability to identify different mobility gestures called human activity recognition (HAR), which includes running, walking, squatting, jumping, and others. The data is collected through an Arduino Nano 33 BLE Sense board with a sampling rate of 119 Hz, which is embedded with an Inertial Measurement Unit (IMU) sensor. The same board is used as a microcontroller to identify human gestures by developing an end-to-end edge computing application. A deep neural network model is trained and then compressed for deployment on the board to create a self-contained, embedded device capable of identifying the type of gesture performed. Three deep learning models, namely Multi-Layer Perceptron (MLP), Convolutional Neural Network – Long Short Term Memory (CNN-LSTM) & CNN-Gated Recurrent Unit (CNN-GRU), are evaluated in the identification of the mobility gestures. The observed accuracy of the models is 96%, 97.1% and 97.8%, respectively, MLP, CNN-LSTM & CNN-GRU across the different gesture categories. The study shows the utility of embedded devices with deep neural network-based models, which can provide low cost, minimal power usage, and meet data privacy requirements in HAR.

Lexical skills and gesture use: A comparison between expressive and receptive/expressive late talkers

BackgroundStudies on late talkers (LTs) highlighted their heterogeneity and the relevance of describing different communicative profiles. AimsTo examine lexical skills and gesture use in expressive (E-LTs) vs. receptive-expressive (R/E-LTs) LTs through a structured task. Methods and proceduresForty-six 30-month-old screened LTs were distinguished into E-LTs (n= 35) and R/E-LTs (n= 11) according to their receptive skills. Lexical skills and gesture use were assessed with a Picture Naming Game by coding answer accuracy (correct, incorrect, no response), modality of expression (spoken, spoken-gestural, gestural), type of gestures (deictic, representational), and spoken-gestural answers’ semantic relationship (complementary, equivalent, supplementary). Outcomes and resultsR/E-LTs showed lower scores than E-LTs for noun and predicate comprehension with fewer correct answers, and production with fewer correct and incorrect answers, and more no responses. R/E-LTs also exhibited lower scores in spoken answers, representational gestures, and equivalent spoken-gestural answers for noun production and in all spoken and gestural answers for predicate production. Conclusions and implicationsFindings highlighted more impaired receptive and expressive lexical skills and lower gesture use in R/E-LTs compared to E-LTs, underlying the relevance of assessing both lexical and gestural skills through a structured task, besides parental questionnaires and developmental scales, to describe LTs’ communicative profiles.

Triboelectric in-sensor deep learning for self-powered gesture recognition toward multifunctional rescue tasks

Gesture recognition is an effective method to voice-off communication in daily life, offering an intuitive and immersive means to interactive terminal. Current commercialized solutions to gesture recognition are exposed to limitations of either susceptibility to environmental conditions or the huge cost on fabrication, energy, and computing power. Here, we propose a deep-learning-enabled smart gesture recognition glove system that can capture and recognize the hand gesture in real time and execute the users’ interactive instructions for multifunctional rescue tasks. The self-powered gesture recognition glove system can successfully classify 7 typical gestures with an accuracy as high as 98.57%. As a proof of concept, a rescue vehicle equipped with water pump, dexterous robotic arm, or laser radar is demonstrated to be guided by gestures outside the environment to complete rescue tasks even in environments that are inaccessible to humans. We aim to use the self-powered gesture recognition glove for future intelligent interaction and intelligent manufacturing, especially in the diversified scenarios that require more advanced and dexterous manipulations in the dark or blocked spaces.

Many morphs: Parsing gesture signals from the noise

Parsing signals from noise is a general problem for signallers and recipients, and for researchers studying communicative systems. Substantial efforts have been invested in comparing how other species encode information and meaning, and how signalling is structured. However, research depends on identifying and discriminating signals that represent meaningful units of analysis. Early approaches to defining signal repertoires applied top-down approaches, classifying cases into predefined signal types. Recently, more labour-intensive methods have taken a bottom-up approach describing detailed features of each signal and clustering cases based on patterns of similarity in multi-dimensional feature-space that were previously undetectable. Nevertheless, it remains essential to assess whether the resulting repertoires are composed of relevant units from the perspective of the species using them, and redefining repertoires when additional data become available. In this paper we provide a framework that takes data from the largest set of wild chimpanzee (Pan troglodytes) gestures currently available, splitting gesture types at a fine scale based on modifying features of gesture expression using latent class analysis (a model-based cluster detection algorithm for categorical variables), and then determining whether this splitting process reduces uncertainty about the goal or community of the gesture. Our method allows different features of interest to be incorporated into the splitting process, providing substantial future flexibility across, for example, species, populations, and levels of signal granularity. Doing so, we provide a powerful tool allowing researchers interested in gestural communication to establish repertoires of relevant units for subsequent analyses within and between systems of communication.

Gestures In Bts Music Video Blood, Sweet, And Tears:Semiotic Approach

This study aims to analyze the semiotic meanings and to find out the types of gestures in the BTS Blood, Sweat, and Tears In theory, Roland Barthes discusses three semiotic meanings: denotative, connotative, and myth. This study uses qualitative methods to examine the substance of the meaning of the phenomenon. Researcher used books, theses, journals, and visual images taken from the BTS Blood, Sweat, and Tears music video as data sources to support analysis. The results of this study: there are 10 data where each data has one to four scenes. From the 10 data, there are types of gestures which include 6 data belonging to the illustrator gesture type, 2 data including the embellishment gesture type, and 2 data belonging to the illustrator gesture type. Of all the semiotics presented, the meaning of this music video is the story of resistance, denial, and eventually acceptance. Once cannot stand in the museum of their life forever and look longingly at the mementos of the past. The world is a place of complex relationships between light and dark, good and evil, virtuosity and sin. As we all cross the threshold of puberty we are faced with the choices that shape our lives. No one person is immune to the temptations of such a life and seeing it, understanding it for what it is, and making those choices is all we can do. No one remains untouched by the darkness.

Semi‐Solid Stretchable Carbon Nanotubes Inkpad for Hand‐Based Haptic Interaction

AbstractDexterity and perceptual abilities of the hand are crucial in virtual reality (VR) interactions. This necessitates haptic gloves capable of accurately measuring finger movements and ensuring comfort, a challenging combination that demands both high sensitivity and stretchability. A flexible sensor fabricated from semi‐solid material CNTs‐Inkpad composed of carbon nanotubes (CNTs) mixed with silica gel is proposed and its real‐time response can be achieved by controlling the ratio. Its semi‐solid form harnesses CNTs' high conductivity and sensitivity, avoiding structural cracking in solid CNTs during extensive stretching, thus the semi‐solid form can adapt to various hand gestures. This mechanism enables the CNTs‐Inkpad to function as strain sensors with high gauge factors (GF = 16.99) and high extensibility (350%). By integrating ten CNTs‐Inkpad sensors and five vibrators, a thin and lightweight haptic glove is developed with high sensitivity and stretchability. The glove enables collaborative measurement of all finger joints required for typical gestures in VR interactions, suitable for motion detection and haptic feedback.

RadarHand: A Wrist-Worn Radar for On-Skin Touch-Based Proprioceptive Gestures

We introduce RadarHand, a wrist-worn wearable with millimetre wave radar that detects on-skin touch-based proprioceptive hand gestures. Radars are robust, private, small, penetrate materials, and require low computation costs. We first evaluated the proprioceptive and tactile perception nature of the back of the hand and found that tapping on the thumb is the least proprioceptive error of all the finger joints, followed by the index finger, middle finger, ring finger, and pinky finger in the eyes-free and high cognitive load situation. Next, we trained deep-learning models for gesture classification. We introduce two types of gestures based on the locations of the back of the hand: generic gestures and discrete gestures. Discrete gestures are gestures that start at specific locations and end at specific locations at the back of the hand, in contrast to generic gestures, which can start anywhere and end anywhere on the back of the hand. Out of 27 gesture group possibilities, we achieved 92% accuracy for a set of seven gestures and 93% accuracy for the set of eight discrete gestures. Finally, we evaluated RadarHand’s performance in real-time under two interaction modes: Active interaction and Reactive interaction. Active interaction is where the user initiates input to achieve the desired output, and reactive interaction is where the device initiates interaction and requires the user to react. We obtained an accuracy of 87% and 74% for active generic and discrete gestures, respectively, as well as 91% and 81.7% for reactive generic and discrete gestures, respectively. We discuss the implications of RadarHand for gesture recognition and directions for future works.

HandFi: WiFi Sensing based Hand Gesture Recognition using Channel State Information

Gesture recognition using WiFi sensing is an emerging and innovative approach that leverages the variations in WiFi signals caused by human hand movements to enable non-contact and intuitive human-computer interaction. The identification of the hand used to perform a gesture significantly impacts the channel state information (CSI) characteristics. It plays a crucial role in data analysis and interpretation. This paper presents a model for classifying the gesture and the hand with which it is performed. We use an Intel 5300 network interface card as a receiver and a commodity WiFi router to collect the CSI values. The CSI values are collected for six left-hand and six right-hand gestures (push left, push right, increase, decrease, stop, and next). The collected CSI values are pre-processed to find the CSI ratio among the sub-carriers. Then four different methods including proposed method are used for classifying the hand and the gesture. The four methods are (1) use of the Hampel filter, (2) use of CSI ratio only, (3) use of correlation matrix along with CSI ratio, and (4) proposed CSI ratio and interquartile distance based method. Then various machine learning techniques such as KNN, Decision trees, and SVM are used to classify the hand (right or left) used for signaling the gestures and the type of gestures. It is found that using the top 30 inter-quartile distance of the CSI-ratios corresponding to the CSI values of the sub-carriers while preserving their positions as the feature vector, the KNN machine learning model can classify the hand and gesture with approximately 99% accuracy for test data. Similar performance is also observed for other performance evaluation parameters like precision and recall.

Representing temporal concepts using redundant gestures in L2 ongoing interactions

Human conversational interaction is multimodal, involving both verbal and non-verbal modalities. That is, when a speaker and listener interact, they use not only spoken messages but also manual gestures. Manual gestures and spoken messages are semantically and temporally related and work together to create and express a complete meaning. This study employs a data-driven approach to investigate how L2 learners spontaneously employ gestures to express temporal concepts in ongoing dyadic interactions using 11 recorded interactions among L2 learners. The distribution and frequency of specific types of manual gestures were examined using sequential and gesture analyses. The results showed that, when representing temporal concepts, the participants produced language-redundant gestures. For example, to convey temporal concepts, they tended to co-express the same information with manual gestures, namely abstract deictic and metaphoric gestures, on an imaginary mental timeline axis, which appeared to represent the English grammatical concepts of tense and aspectual meaning. Regarding the functional differences in gestures in interactions, based on sequential analysis, speakers employed language-redundant gestures to express time concepts explicitly in comprehension sequences and in the negotiation of meaning as a strategy of repair for lexical retrieval, paraphrasing, and clarification. These findings reveal that understanding the use of both modalities, speech and gestures, is critical in uncovering how speakers conceptualize time in their minds and integrate space and time in language.

Mapping out the structural relationship of middle school students’ use of talk and gestures and group outcomes’ quality in collaborative science problem-solving activities

ABSTRACT The current study explores the connection and affordances of talk and gesture in collaborative science problem-solving activities using an emerging analytical approach. A total of 15 three to five-member groups of middle school students participated in a set of clinical collaborative science problem-solving activities. Six groups (three low- and three high-performing groups) were selected for further analyses. The data consisted of successive statements in student conversations that were coded as different types of talk and gestures. Pearson Phi correlation test and Epistemic Network Analysis were used to analyze the data. We found that certain types of gesture were moderately associated with certain types of talk. In addition, we found high-performing groups were significantly different from low-performing groups in the use of higher cognitive engagement and collaborative regulation talks. We did not find a significant difference in the use of gestures on their own. However, we found that when talk and gestures were combined, high-performing groups tended to use higher cognitive engagement and collaborative regulation talks with representing and pointing gestures. These findings support previous studies on the affordances of talk and gesture in collaborative science activities and provide more evidence for how multimodality is important in science learning.

Recurrent gestures in organic chemistry in tertiary education: creating emblems through material and embodied actions

ABSTRACT Background Science teaching and learning, as discursive practices, occur through the utilization of various semiotic modes. Gestures play a pivotal role as an embodied mode that acts in conjunction with speech in oral communication settings, such as classrooms. Previous research on gestures in chemistry and science education has illuminated how interactions involving different types of gestures can enhance learning opportunities for students. However, the investigation of recurrent gestures and their potential function as emblems in classroom discourse remains unexplored. Purpose The objective of this article is to illustrate how catchments or recurrent gestures were constructed, used, and, to some extent, evolved into emblems within the interaction between an organic chemistry teacher and her students throughout an entire semester. Sample, design and methods For the duration of an entire semester, we non-intrusively observed and recorded video footage of one class from an introductory organic chemistry course taught by an experienced university teacher. Our analysis to identify recurrent gestures involved a thorough examination of the lesson recordings. In addition, we scrutinized other semiotic modes that coincided with the recurrent gestures and assessed the metafunction of the words spoken during gesturing. Results During our observation, we noted the occurrence of 23 catchments, with four of them recurring more than 100 times throughout the semester. These four recurrent gestures were closely tied to fundamental concepts considered crucial in the realm of organic chemistry. Remarkably, three out of these four catchments were consistently associated with visual representations. Conclusion Recurrent gestures serve as a means to embody and impart materiality to exceedingly abstract content. By employing such gestures, the teacher managed to associate unfamiliar and abstract domains with familiar and concrete domains. This process of materialization and embodiment aids in perceiving these concepts as tangible entities that can be manipulated in accordance with their own rules.

Gestural Simulation of Motion Verbs

Abstract We aimed to investigate how the path of a motion verb is mentally simulated and realized in gestures when it is encoded in path-joined motion verbs (e.g., “enter the house”), path-disjoined motion verbs (e.g., “go into the house”), and pathless motion verbs (e.g., “walk in the street”). We conducted 2 studies to answer this question. In Study 1, we analyzed gestures produced by presenters in a set of YouTube videos. We used several chi-square tests to find out what type of gesture (path representational, non–path representational, beat, pointing gestures, and no-gesture) co-occurred more frequently with path-joined, path-disjoined, and pathless motion verbs. In Study 2, we analyzed gestures produced by a group of participants in a story-retelling setting. Both studies showed that path representational gestures co-occurred more frequently with path-joined motion verbs than path-disjoined and pathless motion verbs. Furthermore, the probability of using a path representational gesture with a path-joined motion verb having a human subject was higher than that having a nonhuman subject. Although the language difference in Study 1 and Study 2 might be a limitation, the consistency of results of the 2 studies suggests that the findings are generalizable. We suggest that gestural simulation of a motion verb is affected by how the direction or path of motion is encoded in the verb. When the path of motion is encoded in the main part of the verb, the motion and its path or direction are simultaneously simulated. This account implies that the motor system plays an active role in the process of simulating the path.

A Dynamic Model for the Computation of Gesture Types for Image-based Software Agents

<p>Computer vision technology allows the computer to interact with a human operator to quickly complete the interpretation of events and improve the operational workflow of processing events. As computer vision technology continues to evolve, the cost of the equipment continues to increase. Therefore, the stability of the system can be ensured through the design of the middleware and the calculation of the auxiliary functions of the software agents. At present, the recognition of characters for image processing is based on the technology of image recognition, which can provide a more flexible user experience. However, the dilemma of contactless design lies in the processing and calculation of images, which should reduce the inconvenience caused by delays.This article uses a Raspberry Pi as an example of a computing proxy application. After the visual inspection, the verification operation of the software is carried out. Our system is the detection of hand position and movement, and the detection of hand mark position in reconnaissance. In addition, we simultaneously developed management and remote control events and connected to the remote edge computer. After that, we successfully completed the automatic control and serial application of two different edge computing recognition jobs, and verified the image vision computing based on the Raspberry Pi software agent, which can be used for image vision analysis and control applications.</p> <p> </p>

Interaction Design and Research for User Interfaces of Central Bank Digital Currency: An Empirical Study

There is already a consensus among central banks to accelerate the development of Central Bank Digital Currencies (CBDC). To facilitate a smooth money transfer process, the interactive experience of the CBDC user interface design is very important. However, much of the current research has focused on the technical feasibility of the CBDCs, and no scientific research has yet attempted to design an effective CBDC user interface. In this study, a series of CBDC user interfaces were designed to help study the interaction process of CBDCs in a simulated payment transaction context with a focus on the user interface and interactive experience of the CBDCs. The experiment in this study was a 2 (graphical metaphor) × 3 (gesture type) × 2 (visual feedback) mixed factorial design in which a total of 12 CBDC prototypes were created and evaluated based on users’ task performance and a series of subjective evaluations. The results of our empirical study revealed that graphical metaphor user interface design can improve the interface usability and subjective evaluation of CBDCs, and may also trigger the user’s sense of experience with traditional banknotes. Specifically, the use of animated cues for currency movement on the CBDC user interface may help evoke the user’s experience of using cash. In addition, we found that the wave type gesture may cause users to spend more time completing the CBDC operational tasks. Moreover, the animated cue of money moving may have faster performance and a higher degree of rationality. The above-mentioned results may provide an important design reference for CBDC developers to create easy-to-use CBDC services.

Do Blind People Need Gestures to Communicate Effectively?

Gestures are known to play an important role in communication, but to this day it is still not clear what impact visual impairment has on gesture production. The main objective of this article is to investigate: (1) whether the lack of vision reduces (or completely blocks) gesture production in blind individuals, and (2) whether this reduction in (or lack of) gestures has an effect on their abilities to communicate. To address this issue, we performed an experiment in which congenitally blind and sighted people were compared on the number and types of gestures they produced when playing the word-guessing game TABOO. The results obtained confirm that, compared to people who are sighted, blind individuals produce significantly fewer gestures when they communicate. This, however, does not have any negative impact on how effective they are in conveying their messages. The analyses also show that gestures play different roles for blind and sighted people.