Iconic Gestures Research Articles

The information status of iconic enrichments: modelling gradient at-issueness

Abstract Linguistic structures can contribute different types of meaning alongside standard assertions, such as conventional implicatures and presuppositions, which have long been described as being non-at-issue meaning contributions. Although information status has long been handled as a binary opposition between non-at-issue and at-issue content, recent research suggests that a gradient approach may be more appropriate. Building on new – and in the formal linguistic framework so far mostly neglected – data targeting spoken and gestural iconicity, specifically iconic gestures and ideophones, this paper investigates the information status of such iconic contributions in spoken language and suggests a new theoretical concept of at-issueness by spelling it out as a gradient category. The paper highlights a range of factors which can affect the information status of iconic contributions, proposing a scale for iconic phenomena based on these factors. To formally model this scale, we propose an approach in which at-issueness is analysed as a gradient property based on a given structure-inherent at-issueness status and the corresponding proposition’s relevance to a Question Under Discussion in a given context. This analysis accounts for the variations in information status observed between different iconic enrichments and their impact on truth conditions and paves the way for an approach to Common Ground updates using this model. The analysis outlined here allows for a more nuanced understanding of non-at-issue content and its interaction with at-issue content and provides predictions which can guide further experimental work on information status and the factors that influence it.

Interpretations of meaningful and ambiguous hand gestures in autistic and non-autistic adults: A norming study.

Gestures are ubiquitous in human communication, and a growing but inconsistent body of research suggests that people with autism spectrum disorder (ASD) may process co-speech gestures differently from neurotypical individuals. To facilitate research on this topic, we created a database of 162 gesture videos that have been normed for comprehensibility by both autistic and non-autistic raters. These videos portray an actor performing silent gestures that range from highly meaningful (e.g., iconic gestures) to ambiguous or meaningless. Each video was rated for meaningfulness and given a one-word descriptor by 40 autistic and 40 non-autistic adults, and analyses were conducted to assess the level of within- and across-group agreement. Across gestures, the meaningfulness ratings provided by raters with and without ASD correlated at r > 0.90, indicating a very high level of agreement. Overall, autistic raters produced a more diverse set of verbal labels for each gesture than did non-autistic raters. However, measures of within-gesture semantic similarity among the responses provided by each group did not differ, suggesting that increased variability within the ASD group may have occurred at the lexical rather than semantic level. This study is the first to compare gesture naming between autistic and non-autistic individuals, and the resulting dataset is the first gesture stimulus set for which both groups were equally represented in the norming process. This database also has broad applicability to other areas of research related to gesture processing and comprehension. The video database and accompanying norming data are available on the Open Science Framework.

Student Gestures when Solving Local Wisdom-Based Mathematics Problems

When solving math problems, students make spontaneous movements called gestures. The purpose of this research is to find out the variations of gestures that appear when students solve math problems related to rotational geometry based on local wisdom. The qualitative approach used in this research is a descriptive research type. Test questions, documentation results, observations, and interviews were used as research instruments. The subjects of this study were four students in class XI for the 2022/2023 academic year at SMA Negeri Balung. The results showed that the group of students with high abilities used gestures more than students with low abilities. Students with high ability in solving math problems use 11 iconic gestures, 8 metaphoric gestures, and 24 deictic gestures. Meanwhile, the group of students with low abilities used 7 iconic gestures, 6 metaphoric gestures, and 12 deictic gestures. Based on the distribution of the number of student gestures when solving math problems, the gesture that appears the most is the deictic gesture, and the one that appears the least is the metaphoric gesture.

Comparing gesture frequency between autistic and neurotypical individuals: A systematic review and meta-analysis.

While diagnostic assessments for autism routinely screen for reduced frequency of gestures, evidence supporting reduced gesture production in autism is inconsistent. This systematic review and meta-analysis aimed to clarify differences in frequency of gestures between autistic and neurotypical individuals. Included studies compared frequency of gestures between autistic and neurotypical individuals. Database searches (APA PsycInfo, ERIC, MEDLINE, ProQuest Dissertations and Theses Global) and a call for unpublished data on the International Society for Gesture Studies listserv identified research from January 1994 to March 2023. Study quality was assessed using the Joanna Briggs Institute Critical Appraisal Checklist for Quasi-Experimental Designs. Quantitative synthesis involved a narrative review of all findings and meta-analysis of articles allowing effect size calculations, stratified by the type of gesture. Thirty-one articles comparing frequency of gestures between 701 autistic and 860 neurotypical individuals were included in the narrative review, 25 of which were also included in the meta-analysis. Compared to neurotypical individuals, meta-analyses found that autistic individuals produced significantly less total, deictic, and emblematic gestures. While the number of iconic gestures appeared comparable between groups, studies investigating iconic gestures exhibited an almost equal trend of both positive and negative effect sizes, which were mostly nonsignificant. The way gesture production was measured, age, observer familiarity, and task structure (but not overall study quality) moderated the effect size, albeit inconsistently across the types of gestures. Findings have implications relating to profiling gesture use in diagnostic assessments for autism and highlight gaps in our understanding of differences in gesture production in autism. (PsycInfo Database Record (c) 2023 APA, all rights reserved).

Gestures and Their Meanings of Main Character in Daniel’s Movie Everything Everywhere All at Once (2022)

Gesture is a unique nonverbal communication signal, whereby a person conveys a message or expresses himself through conscious or unconscious movements, body movements, and facial expressions" (Vijay, 2013). This study aims to analyze gestures and meaning based on the context of gestures supported by utterances. According to Sugiyono (2013: 13), this research employs a descriptive qualitative methodology. This study uses the theory of McNeil (2005) that classifies gestures into four types, namely Iconic, Deictic, Metaphor, and Beat. The data used in this research is in the form of screenshots taken from the movie "Everything Everywhere All at Once (20"2)" by Daniel. Four types of gestures are found in the movie, as the researcher has done in the analysis. There are 122 data found. There are 11 (9%) data as iconic gestures, 22 (18%) data as deictic gestures, 15 (12%) data as metaphor gestures, and 75 (61%) data as beat gestures. From the data analyzed, the meaning of the gesture is supported by the utterances in dialog (Semantic Meaning). Some gestures meaning is unclear if they are not accompanied or supported by semantic meaning or utterances. Meanwhile, some data do not have a cultural meaning and no relation to the semantic meaning, such as beat type. A beat gesture is a rhythm of speech as a compliment.

Cebir Öğretim Sürecinin Jestler Bağlamında Göstergebilimsel Olarak İncelenmesi

This research aims to examine the gestures used by teachers and students in the algebra learning-teaching process and the purpose of these signs. The participants of this case study are 25 eigth graders and their mathematics teacher. The data were collected through video recordings and clinical interviews. Deductive analysis was used to analyze the data, while synchronic and diachronic analyses were utilized to analyze the signs in terms of semiotics. Findings revealed that the teacher used a bunch of iconic, metaphoric, deictic, and beat gestures during the algebra teaching process. The meaning of the iconic gestures that resulted in the study can be listed as centering and itemization, illustration of a table, constant term, preceding element, decrease, separation, base and power, rectangle and side, length, and vertical-horizontal. On the other hand, the metaphorical gestures attribute holding, equality, moving the term from one side to the other, numbers or letters, equality, distributing and hiding. The deictic gestures imply pointing out boards with fingers, board markers or hands, and indicating students and notebooks with fingers, while the beat gestures refer to lowering the hands from top to bottom, waving the fist up and down and moving the hand back and forth. It is seen that the teacher aims to strengthen her expression and support students' mathematical understanding through the gestures she uses.

The visuo-sensorimotor substrate of co-speech gesture processing

Co-speech gestures are integral to human communication and exhibit diverse forms, each serving a distinct communication function. However, existing literature has focused on individual gesture types, leaving a gap in understanding the comparative neural processing of these diverse forms. To address this, our study investigated the neural processing of two types of iconic gestures: those representing attributes or event knowledge of entity concepts, beat gestures enacting rhythmic manual movements without semantic information, and self-adaptors. During functional magnetic resonance imaging, systematic randomization and attentive observation of video stimuli revealed a general neural substrate for co-speech gesture processing primarily in the bilateral middle temporal and inferior parietal cortices, characterizing visuospatial attention, semantic integration of cross-modal information, and multisensory processing of manual and audiovisual inputs. Specific types of gestures and grooming movements elicited distinct neural responses. Greater activity in the right supramarginal and inferior frontal regions was specific to self-adaptors, and is relevant to the spatiomotor and integrative processing of speech and gestures. The semantic and sensorimotor regions were least active for beat gestures. The processing of attribute gestures was most pronounced in the left posterior middle temporal gyrus upon access to knowledge of entity concepts. This fMRI study illuminated the neural underpinnings of gesture-speech integration and highlighted the differential processing pathways for various co-speech gestures.

Instructors' Gestural Accuracy Affects Geology Learning in Interaction with Students' Spatial Skills.

Complex and often unobservable STEM constructs and processes are represented using a variety of representations, including iconic gestures in which the body is configured or moved to resemble a referent's spatial properties or actions. Earlier researchers have suggested links between gesturing and expertise, leading some to recommend instructional gestures. Earlier research, however, has been largely correlational; furthermore, some gestures may be made with misleading positions or movements. Using the illustrative topic of strike in structural geology, we investigated the existence and impact of inaccurate instructional gestures. In Study 1, we examined videotapes of participants who had been asked to explain strikes to another person. We observed inaccurate (non-horizontal) strike gestures not only among novices (first introduced to strike during the study itself, n = 68) but also among participants who had greater expertise in geology (n = 21). In Study 2, we randomly assigned novices (N = 167) to watch video lessons in which the instructor accompanied verbal explanations of strikes with accurate, inaccurate, or no iconic gestures and tested students' learning on a strike-mapping task. Students with low spatial-perception skills showed no impact of their gestural condition on performance. Students with high spatial-perception skills showed no advantage from accurate gestures but performed significantly worse in the inaccurate-gesture condition. Findings suggest that recommendations to use gestures during instruction should include professional development programs that reduce the occurrence of inaccurate gestures.

A Genealogy of Gestures: On the Nature and Emergence of Forms of Gestural Communication within Shared Routines

Communicative development can be better understood by examining the diversity of gestures and how various forms of gestures are interlinked in their developmental origins – their genealogy. To draw attention to the differences and interrelations among forms of gestures, we group gestures into three families based on their developmental origins: (1) action-based gestures that develop from infants’ spontaneous actions that others respond to; (2) conventional gestures; and (3) iconic gestures. Although these diverse gestures are acquired through somewhat different developmental pathways, we argue that they develop in the context of shared experiences within social routines. What differs is the relative role of the caregiver and child in initiating the routine. In viewing communicative development in this way, we show the importance of basing our investigations on an adequate conception of meaning in order to recognize the similarity in the underlying processes involved in early communicative development.

Iconic gestures serve as primes for both auditory and visual word forms

Abstract Previous studies using cross-modal semantic priming have found that iconic gestures prime target words that are related with the gestures. In the present study, two analogous experiments examined this priming effect presenting prime and targets in high synchrony. In Experiment 1, participants performed an auditory primed lexical decision task where target words (e.g., “push”) and pseudowords had to be discriminated, primed by overlapping iconic gestures that could be semantically related (e.g., moving both hands forward) or not with the words. Experiment 2 was similar but with both gestures and words presented visually. The grammatical category of the words was also manipulated: they were nouns and verbs. It was found that words related to gestures were recognized faster and with fewer errors than the unrelated ones in both experiments and similarly for both types of words.

Multimodal language in child-directed versus adult-directed speech.

Speakers design their multimodal communication according to the needs and knowledge of their interlocutors, phenomenon known as audience design. We use more sophisticated language (e.g., longer sentences with complex grammatical forms) when communicating with adults compared with children. This study investigates how speech and co-speech gestures change in adult-directed speech (ADS) versus child-directed speech (CDS) for three different tasks. Overall, 66 adult participants (Mage = 21.05, 60 female) completed three different tasks (story-reading, storytelling and address description) and they were instructed to pretend to communicate with a child (CDS) or an adult (ADS). We hypothesised that participants would use more complex language, more beat gestures, and less iconic gestures in the ADS compared with the CDS. Results showed that, for CDS, participants used more iconic gestures in the story-reading task and storytelling task compared with ADS. However, participants used more beat gestures in the storytelling task for ADS than CDS. In addition, language complexity did not differ across conditions. Our findings indicate that how speakers employ different types of gestures (iconic vs beat) according to the addressee's needs and across different tasks. Speakers might prefer to use more iconic gestures with children than adults. Results are discussed according to audience design theory.

Iconicity in the emergence of a phonological system?

AbstractIconicity has been described as an impetus for creating sign forms in emerging sign languages and forming signs in established sign languages. Iconic signs are defined as spontaneous or stable signs that directly reflect the representation of their referent. In established sign languages, iconic signs have phonological features. Regarding the link between the motivation for iconic signs and phonological features, we aim to investigate how iconicity might influence the emergence of a phonological system along with the evolution of a new sign language by observing how the rise of a phonological system might be revealed by the evolution of emerging iconic gestures and signs in a new sign language. For this purpose, we inventoried and coded the iconicity nature and phonological structure of 200 signed lexical items collected in two moments of Sao Tome and Principe Sign Language (LGSTP) emergence: at T1 (after 2 years since the deaf habitants initiated their social meetings) and T2 (8 years subsequent to T1 data collection). In the 8 years of LGSTP’s emergence, we found a dominance of iconic signs in tandem with changes in the signs’ internal structure. The handshape is revealed to be the phonological parameter with the greatest development, presenting itself as more complex. The LGSTP lexicon reveals that iconicity seems to prompt the emergence of sign forms. However, iconic strategies remain stable across the evolution of the emergent signs and are independent of the internal structure change of the sign.

Constraints on communicating the order of events in stories through pantomime

Abstract Pantomime is a means of bodily visual communication that is based on iconic gestures that are not fully conventional. It has become a key element in many models of language evolution and a strong candidate for the original human-specific communicative system (Zlatev et al. 2020). Although pantomime affords successful communication in many contexts, it has some semiotic limitations. In this study, we looked at one of them, connected with communicating the order of events in stories. We assumed that pantomime is well-suited for communicating simple stories, where events are arranged in chronological order, and less so for communicating complex stories, where events are arranged in a non-chronological order. To test this assumption, we designed a semiotic game in which participants took turns as directors and matchers. The task of the directors was to mime a story in one of two conditions: chronological or non-chronological; the task of the matchers was to interpret what was mimed. The results showed that the chronological condition was easier for the participants. In the non-chronological condition, we observed that initially, poor communicative success improved as the participants started to use various markers of event order. The results of our study provide insight into the early stages of conventionalisation in bodily visual communication, a potential first step towards protolanguage.

Spontaneous Gestures in L2 Naturalistic Spontaneous Interaction: Effects of Language Proficiency

Gestures produced by language learners have a positive impact on interactions; however, few studies have examined natural conversation data focusing on a learner’s spoken language proficiency level. This study investigates gesture use among learners of English as a second language with varying language proficiency levels (beginner, intermediate, and advanced) to determine whether gesture use and type (e.g., iconic, deictic, metaphoric, and beat gestures) differ by language proficiency level. This study examined 17 video-recorded dyadic interactions in English consisting of mixed-level and same-level pairs. Quantitative analysis followed by a data-driven approach demonstrated that more advanced learners employed gestures with speech more frequently than other groups. During interactions, iconic gestures were used more often by the beginner group, while deictic gestures were employed more by the advanced group. Moreover, the function of the gestures produced by each group during the interactions appeared to be qualitatively varied. These results indicate that gesture use and type may relate to learners’ language profi ciency levels. This study has revealed significant differences in gesture use among learners of English as a second language with varying language proficiency levels, providing insights into learners’ cognition process during verbal communication.

Exploration of the characteristics of teachers' multimodal behaviours in problem‐oriented teaching activities with different response levels

AbstractProblem‐oriented teaching (POT) activities are important in classroom instruction. The level of student response can be influenced by different teacher behaviours in POT. However, the characteristics of teachers' multimodal behaviours at different levels of response are unclear. This study applied epistemic network analysis to explore the characteristics of teachers' multimodal behaviours in POT with different response levels. Using 256 POT segments from 12 classroom teaching videos in primary and secondary schools, this study found that authentic questions and iconic gestures occupied a central position in multimodal behaviours. The concomitant use of multimodal behaviours and the flexibility in eye contact could improve the response levels of students. These findings help understand how teachers' multimodal behaviours can promote students' responses. Practitioner notesWhat is already known about this topic Various behaviours have been shown to impact learners' response, such as teachers' responses to students and teachers' non‐verbal behaviours. Epistemic network analysis can be used to analyse the structure of the connections between cognitive, social and interaction data. What this paper adds This study analyses the characteristics of teachers' multimodal behaviours during problem‐oriented activities. It also compares the salient properties of non‐verbal, verbal and multimodal behavioural networks generated by interactive fragments of students' responses of different cognitive levels. Implications for practice and policy Teachers should ask more authentic questions in the classroom to promote in‐depth thinking processes in students. The appropriate coordination of multimodal behaviours by teachers is crucial to promote the cognitive level of students' responses. Upon making eye contact with students, teachers should also use iconic gestures and be flexible regarding the use of eye contact.

Iconic gestures support novel word learning in multilingual students with SLCN in classrooms

Learning academic vocabulary is a crucial task for all students, but especially challenging for students with speech, language, and communication needs (SLCN) and those who are multilingual. Following a participatory research strategy, we analyze whether iconic gestures can be integrated in teaching routines in an inclusive elementary school and whether the presentation of iconic gestures supports novel word learning over the period of four math lessons. One hundred and sixteen students (44% boys), the majority (91%) with SLCN, participated in the study. We conducted a control group design in eight classes with pre- and post-testing of a target academic vocabulary on the topic “Geometric surfaces and solids.” Results show a significant increase in the acquisition of the receptive and expressive target academic vocabulary for all students, but a predominance of expressive learning performance in favor of students who observed iconic gestures during lessons. Iconic gestures can be easily implemented into teaching, improve novel word learning in students with SLCN and serve as a cuing strategy for naming words in students with severe SLCN.

Multimodal language in bilingual and monolingual children: Gesture production and speech disfluency

AbstractBilingual and monolingual children might have different styles of using multimodal language. This study investigates speech disfluency and gesture production of 5- and 7-year-old Turkish monolingual (N = 61) and Turkish–English bilingual children (N = 51). We examined monolinguals’ Turkish narratives and bilinguals’ Turkish and English narratives. Results indicated that bilinguals were more disfluent than monolinguals, particularly for silent and filled (e.g., umm) pauses. Bilinguals used silent pauses and repetitions (e.g., cat cat) more frequently in English than in Turkish. Gesture use was comparable across language and age groups, except for iconic gestures. Monolinguals produced more iconic gestures than bilinguals. Children's overall gesture frequency predicted disfluency rates only in Turkish. Different gesture types might be orchestrated in the multimodal system, contributing to narrative fluency. The use of disfluency and gesture types might provide insight into bilingual and monolingual children's language development and communication strategies.

Electrophysiological evidence for the enhancement of gesture-speech integration by linguistic predictability during multimodal discourse comprehension

In face-to-face discourse, listeners exploit cues in the input to generate predictions about upcoming words. Moreover, in addition to speech, speakers produce a multitude of visual signals, such as iconic gestures, which listeners readily integrate with incoming words. Previous studies have shown that processing of target words is facilitated when these are embedded in predictable compared to non-predictable discourses and when accompanied by iconic compared to meaningless gestures. In the present study, we investigated the interaction of both factors. We recorded electroencephalogram from 60 Dutch adults while they were watching videos of an actress producing short discourses. The stimuli consisted of an introductory and a target sentence; the latter contained a target noun. Depending on the preceding discourse, the target noun was either predictable or not. Each target noun was paired with an iconic gesture and a gesture that did not convey meaning. In both conditions, gesture presentation in the video was timed such that the gesture stroke slightly preceded the onset of the spoken target by 130 ms. Our ERP analyses revealed independent facilitatory effects for predictable discourses and iconic gestures. However, the interactive effect of both factors demonstrated that target processing (i.e., gesture-speech integration) was facilitated most when targets were part of predictable discourses and accompanied by an iconic gesture. Our results thus suggest a strong intertwinement of linguistic predictability and non-verbal gesture processing where listeners exploit predictive discourse cues to pre-activate verbal and non-verbal representations of upcoming target words.

Mengidentifikasi Gesture Siswa dalam Menyelesaikan Soal Matematika Siswa Kelas VII di SMP N 2 Ampek Angkek

Learning activities are basically communication activities because in the learning process, teachers and students are involved in the process of delivering messages, using media, and receiving messages. Gestures are spontaneous movements of the hands or body which generally occur during face-to-face conversations. The movement is a gesture to convey a message in the form of an impression in a way that words cannot. Gestures are divided into four main categories, namely (1) iconic gestures, (2) metaphoric gestures, (3) deitic gestures and (4) beat gestures. The purpose of this study was to find out what gestures were made by class VII students of SMP N 2 Ampek Angkek during the learning process. The type of research conducted is descriptive qualitative. Sampling of this study used the Proposive Sampling technique. Data collection was carried out by observation, interviews, and video. It was concluded that students used iconic gestures 30 times, deictic gestures 25 times, and metaphoric gestures 5 times when solving math problems in comparative material. The benefits of this gesture are carried out by students who explain material in solving mathematical problems, among others, to focus attention on the intended object, describe the object being thought of, focus attention on the object being discussed, and concretize the idea being thought of. The use of gestures can also make it easier for students or listeners to understand and easily understand the material being studied or the information conveyed.

The role of hand gestures in emotion communication: Do type and size of gestures matter?

We communicate emotions in a multimodal way, yet non-verbal emotion communication is a relatively understudied area of research. In three experiments, we investigated the role of gesture characteristics (e.g., type, size in space) on individuals' processing of emotional content. In Experiment 1, participants were asked to rate the emotional intensity of emotional narratives from the videoclips either with iconic or beat gestures. Participants in the iconic gesture condition rated the emotional intensity higher than participants in the beat gesture condition. In Experiment 2, the size of gestures and its interaction with gesture type were investigated in a within-subjects design. Participants again rated the emotional intensity of emotional narratives from the videoclips. Although individuals overall rated narrow gestures more emotionally intense than wider gestures, no effects of gesture type, or gesture size and type interaction were found. Experiment 3 was conducted to check whether findings of Experiment 2 were due to viewing gestures in all videoclips. We compared the gesture and no gesture (i.e., speech only) conditions and showed that there was not a difference between them on emotional ratings. However, we could not replicate the findings related to gesture size of Experiment 2. Overall, these findings indicate the importance of examining gesture's role in emotional contexts and that different gesture characteristics such as size of gestures can be considered in nonverbal communication.