How are gestures used by politicians? A multimodal co-gesture analysis

Abstract

Gestures are an inseparable part of the language system (McNeill 2005; Kendon 2004), they are semantically co-expressive with speech serving different semantic functions to accompany oral modality (Lin 2017; McNeill 2016). To study these phenomena, we analyse the co-gesture behavior of several Italian politicians during face-to-face interviews. We add a new annotation layer to the PoliModal corpus (Trotta et al. 2020) focused on semantic function of hand movements (Lin 2017; Colletta et al. 2015; Kendon 2004). Then, we explore the patterns of co-occurrence of speech and gestures for the single politicians and from a party perspective. In particular, we address following research questions: i) Are there categories of verbs that systematically accompany hand movements in political interviews? ii) Since the corpus used presents an annotation of "speech constants" (Voghera 2001), is the Lexical Retrieval hypothesis confirmed or are gestures used in correlation with other and different constants of speech? The Lexical Retrieval hypothesis assumes that (a) gesturing occurs during hesitation pauses or in pauses before words indicating problems with lexical retrieval (Dittmann and Llewellyn 1969; Butterworth and Beattie 1978), and (b) that the inability to gesture can cause verbal disfluencies. Finally, we analyse semantic patterns of gesture-speech relationship.