Gestural Communication Research Articles

Gesture's Role in the Communication of Adults With Different Types of Aphasia.

Adults with aphasia gesture more than adults without aphasia. However, less is known about the role of gesture in different discourse contexts for individuals with different types of aphasia. In this study, we asked whether patterns of speech and gesture production of individuals with aphasia vary by aphasia and discourse type and also differ from the speech and gestures produced by adults without aphasia. We compared the amount, diversity, and complexity of speech and gesture production in adults with anomic or Broca's aphasia and adults with no aphasia (n = 20/group) in their first- versus third-person narratives. Adults with Broca's aphasia showed the lowest performance in their amount, diversity, and complexity of speech production, followed by adults with anomic aphasia and adults without aphasia. This pattern was reversed for gesture production. Speech and gesture production also varied by discourse context. Adults with either type of aphasia used a lower amount of and less diverse speech in third-person than in first-person narratives; this pattern was also reversed for gesture production. Overall, our results provide evidence for a compensatory role of gesture in aphasia communication. Adults with Broca's aphasia, who showed the greatest speech production difficulties, also relied most on gesture, and this pattern was particularly pronounced in the third-person narrative context.

“The Poetry of a Dingo’s Bite”

Science fiction has an extensive history of attempting to breach the communication boundary between humans and nonhuman animals by giving nonhuman animals some semblance of human language, with many uplift stories having them speak near-perfect English, their minds being filtered through a human linguistic framework, partly or wholly erasing their voice. Building on the examination of nonhuman animal gestural communication in Brian Massumi’s What Animals Teach Us about Politics (2014), this paper analyses how two works, Ursula K. Le Guin’s “‘The Author of the Acacia Seeds’ and Other Extracts from the Journal of the Association of Therolinguistics” (1974) and Laura Jean McKay’s The Animals in That Country (2020) depict animal behavior in itself as being creative and language-like. Neither story offers a straightforward translation from nonhuman to human, each showing how human linguistic frameworks leave gaps for the untranslatable complexities in nonhuman animal gestures. This I suggest shows that further exploration of nonhuman animal communication in science fiction can allow us to move beyond ideas of human exceptionalism and logocentrism and can turn the hierarchical scale of communication into more of a spectrum with various communication types.

Task-related differences in the gesture production of young autistic children.

In typically developing (TD) children, gesture emerges around 9months of age, allowing children to communicate prior to speech. Due to the important role gesture plays in the early communication of autistic and TD children, various tasks have been used to assess gesture ability. However, few data exist on whether and how tasks differentially elicit gesture, particularly for samples of racially and ethnically diverse autistic children. In this study, we explored if task (a naturalistic parent-child interaction [NPCI]; structured assessment of child communication) differentially elicited rate or type of gesture production for young autistic children. This secondary analysis included baseline data from 80 racially and ethnically diverse autistic children aged 18-59months who participated in one of two larger studies. Video recordings of NPCIs and an assessment of child communication with standardised administration procedures were collected at baseline. Child gesture rate (number of gestures produced per 10min) and type were extracted from these recordings and analysed. The structured assessment elicited more gestures than the NPCI. In terms of gesture type, points, gives, and reaches accounted for 76% of child gestures. Points (which are developmentally more advanced than reaches and gives) were produced at the highest rates within book exploration. Distal points (which are more developmentally advanced than proximal or contact points) were produced at the highest rates when children were tempted to request. Our findings indicate elicitation tasks differentially elicit type and rate of gesture for young autistic children. To assess the gesture production of young autistic children, a structured task designed to elicit child requests will probe the developmental sophistication of the child's gesture repertoire, eliciting both the most gestures and the most developmentally advanced gestures. What is already known on the subject Because of the importance of gesture in early communication for autistic and typically developing children, various tasks have been used to assess it. However, little is known about whether tasks differentially elicit type or rate of gesture for young autistic children from diverse racial and ethnic backgrounds. What this paper adds to existing knowledge Elicitation tasks differentially elicit type and rate of gesture for young autistic children in the early stages of gesture. What are the potential or actual clinical implications of this work? We recommend a structured task designed to elicit child requests to assess the developmental sophistication of a child's gesture repertoire.

The Role of Gesture in Multimodal Communication

Purpose: The general objective of the study was to examine the role of gesture in multimodal communication.
 Methodology: The study adopted a desktop research methodology. Desk research refers to secondary data or that which can be collected without fieldwork. Desk research is basically involved in collecting data from existing resources hence it is often considered a low cost technique as compared to field research, as the main cost is involved in executive’s time, telephone charges and directories. Thus, the study relied on already published studies, reports and statistics. This secondary data was easily accessed through the online journals and library.
 Findings: The findings reveal that there exists a contextual and methodological gap relating to the role of gesture in multimodal communication. Preliminary empirical review revealed that that gestures play a crucial role in multimodal communication, serving as more than mere supplements to verbal discourse. Through theoretical frameworks like social semiotics and empirical research across various contexts, it was found that gestures enhance comprehension, memory retention, and social interactions. The findings emphasized the importance of recognizing and leveraging gestures in communication, urging educators, communicators, and policymakers to develop strategies that harness the full potential of multimodal communication for fostering understanding and connection in diverse settings.
 Unique Contribution to Theory, Practice and Policy: The Cognitive Load theory, Symbolic Interactionism and Social Semiotics model may be used to anchor future studies on the role of gestures in multimodal communication. The study recommended integrating gesture theories within existing communication frameworks, enriching theoretical discourse and offering new perspectives on multimodal communication dynamics. Practically, it advocated for incorporating gesture-based interventions in education and professional settings to enhance communication effectiveness and build rapport. At the policy level, it called for the integration of multimodal communication strategies into educational curricula and workplace training programs to support inclusive and effective communication practices. The study also suggested further research into the cognitive mechanisms of gesture production and comprehension, alongside the development of technology-mediated communication tools and guidelines for inclusive communication practices. Additionally, it proposed interdisciplinary collaborations to advance understanding of multimodal communication processes and drive innovation in the field.

LERMO: A Novel Web Game for AI-Enhanced Sign Language Recognition

Sign language is a visual and gestural communication system used by deaf and hearing-impaired people. Despite numerous deep learning methods proposed for automatic interpretation, a gap persists in developing applications that effectively utilize these models for assisting sign language studies and inclusion. We introduce LERMO (https://lermo.app/), a web game merging machine learning and gamification to enhance sign language fingerspelling. Inspired by Wordle™, LERMO offers an interactive word-guessing game where users can play using a video camera. We create a new dataset of labeled landmark fingerspelling and design our model to ensure optimal speed and efficiency to run on a web browser. We survey approximately 40 users, which find LERMO user-friendly and innovative. From those, 95% believe LERMO could be used to enhance fingerspelling skills.

Exploring Sign Language Detection on Smartphones: A Systematic Review of Machine and Deep Learning Approaches

In this modern era of technology, most of the accessibility issues are handled with the help of smart devices and cutting-edge gadgets. Smartphones play a crucial role in addressing various accessibility challenges, including voice recognition, sign language detection and interpretation, navigation systems, speech-to-text conversion, and vice versa, among others. They are computationally powerful enough to handle and run numerous machine and deep learning applications. Among various accessibility challenges, speech disorders represent a disability where individuals struggle to communicate verbally. Similarly, hearing loss is a disability that impairs an individual’s ability to hear, necessitating reliance on gestures for communication. A significant challenge encountered by people with speech disorders, hearing loss, or both is their inability to effectively convey or receive messages from others. Hence, these individuals heavily depend on the sign language (a gesture-based communication) method, typically involving hand movements and expressions. To the best of our knowledge, there are currently no comprehensive review and/or survey articles available that cover the literature on speech disabilities and sign language detection and interpretation via smartphones utilizing machine learning and/or deep learning approaches. This study fills the gap in the literature by analyzing research publications on speech disabilities, published from 2012 to July 2023. A rigorous search and standard strategy for formulating the literature along with a well-defined theoretical framework for results and findings have been used. The paper has implications for practitioners and researchers working in accessibilities in general and smart/intelligent gadgets and applications for speech-disabled people in specific.

Many morphs: Parsing gesture signals from the noise

Parsing signals from noise is a general problem for signallers and recipients, and for researchers studying communicative systems. Substantial efforts have been invested in comparing how other species encode information and meaning, and how signalling is structured. However, research depends on identifying and discriminating signals that represent meaningful units of analysis. Early approaches to defining signal repertoires applied top-down approaches, classifying cases into predefined signal types. Recently, more labour-intensive methods have taken a bottom-up approach describing detailed features of each signal and clustering cases based on patterns of similarity in multi-dimensional feature-space that were previously undetectable. Nevertheless, it remains essential to assess whether the resulting repertoires are composed of relevant units from the perspective of the species using them, and redefining repertoires when additional data become available. In this paper we provide a framework that takes data from the largest set of wild chimpanzee (Pan troglodytes) gestures currently available, splitting gesture types at a fine scale based on modifying features of gesture expression using latent class analysis (a model-based cluster detection algorithm for categorical variables), and then determining whether this splitting process reduces uncertainty about the goal or community of the gesture. Our method allows different features of interest to be incorporated into the splitting process, providing substantial future flexibility across, for example, species, populations, and levels of signal granularity. Doing so, we provide a powerful tool allowing researchers interested in gestural communication to establish repertoires of relevant units for subsequent analyses within and between systems of communication.

Using LSTM to translate Thai sign language to text in real time

Between 2019 and 2022, as the Covid-19 pandemic unfolded, numerous countries implemented lockdown policies, leading most corporate companies to permit employees to work from home. Communication and meetings transitioned to online platforms, replacing face-to-face interactions. This shift posed challenges for deaf or hearing-impaired individuals who rely on sign language, using hand gestures for communication. However, it also affected those who can hear clearly but lack knowledge of sign language. Unfortunately, many online meeting platforms lack sign language translation features. This study addresses this issue, focusing on Thai sign language. The objective is to develop a model capable of translating Thai sign language in real-time. The Long Short-Term Memory (LSTM) architecture is employed in conjunction with MediaPipe Holistic for data collection. MediaPipe Holistic captures keypoints of hand, pose, and head, while the LSTM model translates hand gestures into a sequence of words. The model’s efficiency is assessed based on accuracy, with real-time testing achieving an 86% accuracy, slightly lower than the performance on the test dataset. Nonetheless, there is room for improvement, such as expanding the dataset by collecting data from diverse individuals, employing data augmentation techniques, and incorporating an attention mechanism to enhance model accuracy.

Shared semantics: Exploring the interface between human and chimpanzee gestural communication

Striking similarities across ape gestural repertoires suggest shared phylogenetic origins that likely provided a foundation for the emergence of language. We pilot a novel approach for exploring possible semantic universals across human and nonhuman ape species. In a forced‐choice task, n = 300 participants watched 10 chimpanzee gesture forms performed by a human and chose from responses that paralleled inferred meanings for chimpanzee gestures. Participants agreed on a single meaning for nine gesture forms; in six of these the agreed form‐meaning pair response(s) matched those established for chimpanzees. Such shared understanding suggests apes' (including humans') gesturing shares deep evolutionary origins.

Windows into spatial cognition: Mechanisms by which gesture-based instruction improve anatomy learning.

The ability to create efficient "mental models" or representations of anatomical structures is crucial for achieving competence in most areas of anatomy. Gesture-based teaching has been recognized to lighten cognitive loads and allow superior mental model creation compared to non-gestural teaching practices. This commentary explores the cognitive basis and possible mechanisms behind this advantage such as (1) reducing visual working memory load, (2) allowing parallel and sequential development of internal representations, and (3) facilitating preferential feature extraction and improved organization of spatial information. We also highlight how information transfer limitations of the gestural medium, interestingly, unveil features and organizational motifs preserved in the "expert's" mental schemas concerning particular anatomical structures. The universal and innate use of gestures in communication, their visual nature, and the ability to break down complex spatial information through sequential steps, all add to the immense potential of this subtle yet powerful tool of hand gestures. As pedagogical practices in the anatomical sciences continue to evolve largely towards technology-enhanced teaching utilizing perceptually richer media, the unique advantages of gesture-based teaching need to be reemphasized.

The Role of Life History and Familiarity in Performance of Working and Non-Working Dogs (Canis lupus familiaris) in a Point-Following Task.

Domestic dogs are very successful at following human communicative gestures in paradigms such as the object-choice task. Pet dogs also prefer responding to cues given by a familiar cue-giver and this had not been found in working dogs. Therefore, we tested three groups of dogs in the object-choice task (n = 54): the groups were "Actively working" dogs from working dog breeds, pet dogs from "Non-working breeds" and pet dogs from "Working breeds". We found that "Actively working" and "Working breeds" dog groups outperformed "Non-working breeds" in following a point in the object-choice task. We also found that both "Actively working" and "Working breeds" preferred a familiar cue-giver over an unfamiliar one, in contrast to previous findings. Therefore, we conclude that dogs' abilities to perform well in the object-choice task is influenced by the selective history of the breed, and this is then increased by life experience and training.

THE ORIGIN AND MEANING OF OLD TURKIC WORDS

Language has been the primary means of communication since the creation of mankind. Gesture communication of the first people and then language evolved along with civilization, forming languages through divisions such as geographical environment and political class. Language is a living organism being born anew, developing, and then dying, disappearing from the surface of the earth. As an argument, the Latin language can be mentioned, which plays a key historical role. Although it is a dead language, the languages separated from the Latin branch are a reflection of European languages today. The article examines the specific lexical meaning and etymology of some incorrectly translated words or words with different meaning used in Old Turkic runic written monuments. In addition, when considering the research material of the Old Turkic language, they were compared based on the materials of the Middle Turkic and modern Turkic languages. It is mainly compared with the lexical fund and grammar of the Kazakh language and is considered as the main connecting language. Key words: Orkhon-Yenisei, Turkic, runes, etymology, Kazakh language, translation, written monuments.

Emoji as gesture in digital communication: Emoji improve comprehension of indirect speech.

As reliance on digital communication grows, so does the importance of communicating effectively with text. Yet when communicating with text, benefits from other channels, such as hand gesture, are diminished. Hand gestures support comprehension and disambiguate characteristics of the spoken message by providing information in a visual channel supporting speech. Can emoji (pictures used to supplement text communication) perform similar functions? Here, we ask whether emoji improve comprehension of indirect speech. Indirect speech is ambiguous, and appropriate comprehension depends on the receiver decoding context cues, such as hand gesture. We adapted gesture conditions from prior research (Kelly et al.,1999, Experiment 2) to a digital, text-based format, using emoji rather than gestures. Participants interpreted 12 hypothetical text-message exchanges that ended with indirect speech, communicated via text only, text+emoji, or emoji only, in a between-subjects design. Like that previously seen for hand gesture, emoji improved comprehension. Participants were more likely to correctly interpret indirect speech in the emoji-only condition compared with the text+emoji and the text-only conditions, and more likely in the text+emoji condition compared to the text-only condition. Thus, emoji are not mere decoration, but rather are integrated with text to communicate and disambiguate complex messages. Similar to gesture in face-to-face communication, emoji improve comprehension during text-based communication.

Use of stories and metacognitive strategies to promote positive school life

Developing personal and social skills in a school environment is indispensable in the learning process. Therefore, the objective of this article is to present a literature review of the use of storytelling and metacognitive strategies to promote positive school coexistence. The research is basic, with hermeneutic phenomenological design, literary review technique, and card use to register fifty-four scientific articles consulted from Scielo, Redalyc, ProQuest, and Dialnet since 2019. The results show the story as a didactic resource that favors reading comprehension and facilitates written, verbal, and gestural communication, developing creativity and integration capacity with metacognitive strategies that promote self-regulation and social cohesion bonds. In conclusion, a positive school coexistence is based on inclusive relationships and respectful language where diversity and different personal and social skills are valued under harmonious environments that strengthen socialization.

SDViT: Stacking of Distilled Vision Transformers for Hand Gesture Recognition

Hand gesture recognition (HGR) is a rapidly evolving field with the potential to revolutionize human–computer interactions by enabling machines to interpret and understand human gestures for intuitive communication and control. However, HGR faces challenges such as the high similarity of hand gestures, real-time performance, and model generalization. To address these challenges, this paper proposes the stacking of distilled vision transformers, referred to as SDViT, for hand gesture recognition. An initially pretrained vision transformer (ViT) featuring a self-attention mechanism is introduced to effectively capture intricate connections among image patches, thereby enhancing its capability to handle the challenge of high similarity between hand gestures. Subsequently, knowledge distillation is proposed to compress the ViT model and improve model generalization. Multiple distilled ViTs are then stacked to achieve higher predictive performance and reduce overfitting. The proposed SDViT model achieves a promising performance on three benchmark datasets for hand gesture recognition: the American Sign Language (ASL) dataset, the ASL with digits dataset, and the National University of Singapore (NUS) hand gesture dataset. The accuracies achieved on these datasets are 100.00%, 99.60%, and 100.00%, respectively.

Smart-Data-Glove-Based Gesture Recognition for Amphibious Communication.

This study has designed and developed a smart data glove based on five-channel flexible capacitive stretch sensors and a six-axis inertial measurement unit (IMU) to recognize 25 static hand gestures and ten dynamic hand gestures for amphibious communication. The five-channel flexible capacitive sensors are fabricated on a glove to capture finger motion data in order to recognize static hand gestures and integrated with six-axis IMU data to recognize dynamic gestures. This study also proposes a novel amphibious hierarchical gesture recognition (AHGR) model. This model can adaptively switch between large complex and lightweight gesture recognition models based on environmental changes to ensure gesture recognition accuracy and effectiveness. The large complex model is based on the proposed SqueezeNet-BiLSTM algorithm, specially designed for the land environment, which will use all the sensory data captured from the smart data glove to recognize dynamic gestures, achieving a recognition accuracy of 98.21%. The lightweight stochastic singular value decomposition (SVD)-optimized spectral clustering gesture recognition algorithm for underwater environments that will perform direct inference on the glove-end side can reach an accuracy of 98.35%. This study also proposes a domain separation network (DSN)-based gesture recognition transfer model that ensures a 94% recognition accuracy for new users and new glove devices.

What we know and don't know about great ape cultural communication in the wild.

Following the first descriptions of culture in primates, widespread agreement has developed that the term can be applied to nonhumans as group-specific, socially learned behaviors. While behaviors such as those involving extractive tool use have been researched intensively, we propose that behaviors that are more subtle, less likely to be ecologically constrained, and more likely to be socially shaped, such as cultural forms of communication, provide compelling evidence of culture in nonhuman primates. Additionally, cultural forms of communication can provide novel insights into animal cognition such as the capacity for conformity, conventionalized meanings, arbitrariness in signal forms, and even symbolism. In this paper we focus on evidence from studies conducted on wild great apes. First, we provide a thorough review of what exactly we do know, and by extension don't know, about great ape cultural communication. We argue that detailed research on both vocal and gestural communication in wild great apes shows a more nuanced and variable repertoire than once assumed, with increasing support for group-specific variation. Second, we discuss the relevance of great ape cultural communication and its potential for illustrating evolutionary continuity for human-like cultural attributes, namely cumulative culture and symbolism. In sum, a concerted effort to examine cultural forms of communication in great apes could reveal novel evidence for cultural capacities that have thus far been heavily debated in the literature and can simultaneously contribute to an improved understanding of the complex minds of our closest living relatives.

The Role of Gestures in Communication

Because it plays such a very significant role in the life of every human being, it is difficult to imagine a world without communication. After all, communication, both verbally and nonverbally, has occurred and evolved between people since the dawn of humankind. As it is, during communicative interaction, verbal contact is supplemented with non-verbal information, laying the foundation for full-fledged communication. That means that nonverbal communication is an essential aspect of speech communication because it can either greatly increase the semantic meaning of a word or significantly weaken it. Moreover, non-verbal means of communication are typically expressive and laconic and greatly increase the possibilities of communication between people. That being said, one of the most commonly used non-verbal communications is gestures. In communication, gestures accompany speech or replace it. Gestures include movement of the hands, face, or other body parts, and each person uses various motions in communication to convey his thoughts. The gestures we use in daily communication add to a language’s vocabulary and syntax. Furthermore, a gesture is more expressive than a unit of verbal language. With this in mind, this article discusses the various aspects of non-verbal communication and provides examples of typical gestures a person uses in everyday communication. This study consisted of 50 students ranging in age from 17 to 25 years old and aimed to reveal what types of gestures are used much more frequently by young people in everyday life.

Headar

Nod and shake of one's head are intuitive and universal gestures in communication. As smartwatches become increasingly intelligent through advances in user activity sensing technologies, many use scenarios of smartwatches demand quick responses from users in confirmation dialogs, to accept or dismiss proposed actions. Such proposed actions include making emergency calls, taking service recommendations, and starting or stopping exercise timers. Head gestures in these scenarios could be preferable to touch interactions for being hands-free and easy to perform. We propose Headar to recognize these gestures on smartwatches using wearable millimeter wave sensing. We first surveyed head gestures to understand how they are performed in conversational settings. We then investigated positions and orientations to which users raise their smartwatches. Insights from these studies guided the implementation of Headar. Additionally, we conducted modeling and simulation to verify our sensing principle. We developed a real-time sensing and inference pipeline using contemporary deep learning techniques, and proved the feasibility of our proposed approach with a user study (n=15) and a live test (n=8). Our evaluation yielded an average accuracy of 84.0% in the user study across 9 classes including nod and shake as well as seven other signals -- still, speech, touch interaction, and four non-gestural head motions (i.e., head up, left, right, and down). Furthermore, we obtained an accuracy of 72.6% in the live test which reveals rich insights into the performance of our approach in various realistic conditions.

Gesture, meaning, and intentionality: from radical to pragmatist enactive theory of language

AbstractThe article argues in favour of a pragmatist enactive interpretation of the emergence of the symbolic and contentful mind from a basic form of social communicative interaction in which basic cognitive capacities are involved. Through a critical overview of Radical Enactivists (RECers)’ view about language, the article focuses on Mead’s pragmatist behavioural theory of meaning that refers to the gestural conversation as the origin of the evolution of linguistic conversation. The article develops as follows. After exposing the main elements of REC’s theory of cognition and language that involve the construction of a theory of natural signs (teleosemiotics) and basic directionality (Ur-intentionality), some critical points of Hutto and Myin’s proposal will be highlighted. To foster a continuist perspective of language, the behavioural theory of meaning and language that Mead develops from the notion of gesture will be analysed. His theory is akin to REC and could augment the bare bones of REC’s sketched perspective, helping to include Ur-intentionality in a broader non-dualistic phylogenetic and ontogenetic theory of symbolic language from gestural communication, thus helping to overcome the distinction between a content-less intentionality and a content-involving intentionality, i.e., a semantic propositional intentionality. Furthermore, a recent revival of Mead’s theory testifies to its up-to-date relevance to explain the innate social dimension of human and non-human animals, and the human communicative capacity through the conditioning of bio-social canons and structures.