Alignment Of Gestures Research Articles

Gestural Alignment in Spoken Simultaneous Interpreting: A Mixed-Methods Approach

Cognitive and behavioral alignment plays a major role in simultaneous interpreting, the interpreter centrally monitoring and accommodating his/her behavior to that of the speaker-source. In parallel, the place of gesture in the interpreters’ practice, as well as its degree of convergence with respect to the gestures of the speaker-source, has been scarcely analyzed until very recently. The multimodal data for this study were collected under (quasi-)experimental conditions in a real court interpreting setting during spoken training exercises performed by two novice interpreters. In this exploratory study, the gestural performance of the interpreters, including their degree of gestural alignment towards the speaker-source, is analyzed and compared using a mixed-methods approach to a randomized sample of the recorded data. The analysis combines a basic descriptive quantification of body movements and a qualitative and comparative analysis of the gesture types performed by the speaker-source and the interpreters. The results show that, in spite of individual differences in interpreting fluency and gestural styles, both interpreters tend to align with the speaker-source’s gestural behavior in several ways, and thus a basic taxonomy of gestural convergence between the speaker-source and the interpreters is defined according to several criteria (mainly, gesture presence and gesture type). Our conclusions also allow us to formulate new research questions and hypotheses to be tested in future studies (e.g., types of gestures by the speaker-source that prompt a higher degree of alignment).

The Primacy of Multimodal Alignment in Converging on Shared Symbols for Novel Referents

ABSTRACT When people interact to establish shared symbols for novel objects or concepts, they often rely on multiple communicative modalities as well as on alignment (i.e., cross-participant repetition of communicative behavior). Yet these interactional resources have rarely been studied together, so little is known about if and how people combine multiple modalities in alignment to achieve joint reference. To investigate this, we systematically track the emergence of lexical and gestural alignment in a referential communication task with novel objects. Quantitative analyses reveal that people frequently use a combination of lexical and gestural alignment, and that such multimodal alignment tends to emerge earlier compared to unimodal alignment. Qualitative analyses of the interactional contexts in which alignment emerges reveal how people flexibly deploy lexical and gestural alignment in line with modality affordances and communicative needs.

Alignment in Multimodal Interaction: An Integrative Framework.

When people are engaged in social interaction, they can repeat aspects of each other’s communicative behavior, such as words or gestures. This kind of behavioral alignment has been studied across a wide range of disciplines and has been accounted for by diverging theories. In this paper, we review various operationalizations of lexical and gestural alignment. We reveal that scholars have fundamentally different takes on when and how behavior is considered to be aligned, which makes it difficult to compare findings and draw uniform conclusions. Furthermore, we show that scholars tend to focus on one particular dimension of alignment (traditionally, whether two instances of behavior overlap in form), while other dimensions remain understudied. This hampers theory testing and building, which requires a well‐defined account of the factors that are central to or might enhance alignment. To capture the complex nature of alignment, we identify five key dimensions to formalize the relationship between any pair of behavior: time, sequence, meaning, form, and modality. We show how assumptions regarding the underlying mechanism of alignment (placed along the continuum of priming vs. grounding) pattern together with operationalizations in terms of the five dimensions. This integrative framework can help researchers in the field of alignment and related phenomena (including behavior matching, mimicry, entrainment, and accommodation) to formulate their hypotheses and operationalizations in a more transparent and systematic manner. The framework also enables us to discover unexplored research avenues and derive new hypotheses regarding alignment.

Gesture in contexts of scopal ambiguity: Negation and quantification in English

AbstractGestures can play a facilitative role in the interpretation of structural ambiguities (Guellaiï, Langus, & Nespor, 2014; Prieto, Borràs-Comes, Tubau, & Espinal, 2013; Tubau, González-Fuente, Prieto, & Espinal, 2015) and are associated with spoken expression of negation (Calbris, 2011; Harrison, 2014a; Kendon, 2002, 2004). This study examines gestural forms and timing patterns with specific interpretations intended by speakers in a context of negation in English where the presence of quantification (all/most/many) yields scope ambiguities, for example,All the students didn’t go= (1). Some number of the students went, but all is not the correct number (negation takes wide scope over the quantifier;not>all), versus (2) some number of the students didn’t go, and all is that number (negation takes narrow scope over the quantifier,all>not; see Horn, 2001, Jackendoff, 1972; Syrett, Simon, & Nisula, 2014b). Twenty-five native English speakers produced scopally ambiguous sentences. Analyses of 317 co-occurring gestures revealed a preponderance of head gestures and use of semantically congruent head shakes, alignment of gestures with the negator, and lengthening of gesture strokes where interpretations involved narrow-scope negation. Results are discussed with reference to scope of negation and gesture (Harrison, 2010, 2013, 2014a, 2014b) particularly in comparison to variable patterns found for prosody (Syrett, Simon, & Nisula, 2014a).

Aspects of gestural alignment in task-oriented dialogues

Aspects of gestural alignment in task-oriented dialoguesInterlocutors in a conversation influence each other in a number of dimensions. This process may lead to observable changes in their communicative behaviour. The directions and profiles of these changes are often correlated with the quality of interaction and may predict its success. In the present study, the gestural component of communication is scrutinised for changes that may reflect the process of alignment. Two types of task-oriented dialogues between teenagers are recorded and annotated for gestures and their features. We hypothesize that the dialogue task type (collaborative vs. competitive), as well as certain culture-specific properties of alignment that differ between German and Polish pairs, may significantly influence the process of communication. In order to explore the data and detect tendencies in gestural behaviour, automatised annotation mining and statistical exploration have been used, including a moving frame approach aimed at the investigation of co-occurring strokes as well as re-occurring strokes and their features. Significant differences between German and Polish speakers, as well as between the two dialogue types, have been found in the number of gestures, stroke duration and amplitude. Aspekty wzajemnego dopasowania gestykulacji w dialogach zadaniowychUczestnicy konwersacji wpływają na siebie wzajemnie w wielu różnych wymiarach. Proces ten może prowadzić do obserwowalnych zmian w ich zachowaniach komunikacyjnych. Kierunek i charakter tych zmian często wiążą się z jakością interakcji i mogą pozwolić przewidzieć jej sukces. Celem niniejszego badania jest eksploracja gestowego składnika komunikacji pod kątem zjawisk, które mogą odzwierciedlać zjawisko wzajemnego dopasowania uczestników. Zarejestrowano dwa typy dialogów zadaniowych z udziałem nastolatków. Nagrania anotowano pod kątem gestów i ich cech. W badaniu postawiono hipotezę, iż typ zadania dialogowego (ukierunkowane na współpracę vs. na konkurencję), jak również pewne specyficzne cechy kulturowe dopasowania komunikacyjnego, które różnicują polskie i niemieckie pary rozmówców, mogą oddziaływać na proces komunikacji. W celu eksploracji danych oraz wykrycia tendencji cechujących zachowania gestowe, wykorzystano zautomatyzowane metody analizy anotacji oraz techniki statystyczne. Uwzględniono podejście oparte na ruchomych oknach czasowych, umożliwiające analizę współwystępowania jak i powtarzania gestów właściwych i ich właściwości. Wykazano istotne różnice między rozmówcami polskimi i niemieckimi oraz między typami zadań dialogowych w zakresie liczby gestów, czasu trwania oraz amplitudy gestu właściwego.

What you see is what you do: on the relationship between gaze and gesture in multimodal alignment

abstractInteractive language use inherently involves a process of coordination, which often leads to matching behaviour between interlocutors in different semiotic channels. We study this process of interactive alignment from a multimodal perspective: using data from head-mounted eye-trackers in a corpus of face-to-face conversations, we measure which effect gaze fixations by speakers (on their own gestures, condition 1) and fixations by interlocutors (on the gestures by those speakers, condition 2) have on subsequent gesture production by those interlocutors. The results show there is a significant effect of interlocutor gaze (condition 2), but not of speaker gaze (condition 1) on the amount of gestural alignment, with an interaction between the conditions.

Exploring the Alignment Space – Lexical and Gestural Alignment with Real and Virtual Humans

Communication is characterized by speakers' dynamic adaptations and coordination of both linguistic and nonverbal behaviors. Understanding this phenomenon of alignment and its underlying mechanisms and processes in both human-human and human-computer interaction is of particular importance when building artificial interlocutors. In this paper we contribute to the exploration of the still fragmentary alignment space with two controlled experiments on lexical and gestural alignment. Our results show, on the one hand, that human interlocutors align their lexical choice and gesture handedness in similar ways when interacting with real versus virtual humans. On the other hand we found, however, also subtle differences. We discuss our findings in terms of a dynamic interplay of multiple components (automatic and strategic) and mechanisms (communicative and social) of alignment.

The varieties of remembered experience: Moving memory beyond the bounded self

We review the contributions to this Special Issue that highlight the diverse ways in which memory takes place that go beyond the standard personal autobiographical memory and its reliance on internal imagery. We look at how contributors explore a highly individual memory of trauma and re-consider it as a complex, socially contested phenomenon. We next turn to a discussion of shared memory within dyads and then look at a contribution that examines bodily and gestural alignment during shared recollection among group members and/or families. From there, contributors raise considerations of collective memory in prisoner-of-war survivors and among football fans attending a World Cup event. The next contribution illustrates how collective forgetting creates social bonds in a similar manner to collective remembering. Finally, we show how the boundaries of memory are being stretched by digital technology through its influence on how we recall and share memories. Methodological innovations are also discussed.

Capture, Modeling, and Recognition of Expert Technical Gestures in Wheel-Throwing Art of Pottery

This research has been conducted in the context of the ArtiMuse project that aims at the modeling and renewal of rare gestural knowledge and skills involved in the traditional craftsmanship and more precisely in the art of wheel-throwing pottery. These knowledge and skills constitute intangible cultural heritage and refer to the fruit of diverse expertise founded and propagated over the centuries thanks to the ingeniousness of the gesture and the creativity of the human spirit. Nowadays, this expertise is very often threatened with disappearance because of the difficulty to resist globalization and the fact that most of those “expertise holders” are not easily accessible due to geographical or other constraints. In this article, a methodological framework for capturing and modeling gestural knowledge and skills in wheel-throwing pottery is proposed. It is based on capturing gestures using wireless inertial sensors and statistical modeling. In particular, we used a system that allows for online alignment of gestures using a modified Hidden Markov Model. This methodology is implemented into a human--computer interface, which permits both the modeling and recognition of expert technical gestures. This system could be used to assist in the learning of these gestures by giving continuous feedback in real time by measuring the difference between expert and learner gestures. The system has been tested and evaluated on different potters with rare expertise, which is strongly related to their local identity.