Abstract
This paper was written with the aim of highlighting the functional and structural correlations between gesticulation and prosody, focusing on gesture / prosody synchronization in spontaneous spoken Italian. The gesture annotation used follows the LASG model (Bressem et al. 2013), while the prosodic annotation focuses on the identification of terminal and non-terminal prosodic breaks which, according to L-AcT (Cresti, 2000; Moneglia & Raso 2014), determine speech act boundaries and the information structure, respectively. Gesticulation co-occurs with speech in about 90% of the speech flow examined and gestural arcs are synchronous with prosodic boundaries. Gesture Phrases, which contain the expressive phase (Stroke) never cross terminal prosodic boundaries, finding in the utterance the maximum unit for gesture / speech correlation. Strokes may correlate with all information unit types, however only infrequently with Dialogic Units (i.e. those functional to the management of the communication). The identification of linguistic units via the marking of prosodic boundaries allows us to understand the linguistic scope of the gesture, supporting its interpretation. Gestures may be linked at different linguistic levels, namely those of: a) the word level; b) the information unit phrase; c) the information unit function; d) the illocutionary value.
Highlights
The connection between gesture and prosody is considered fundamental to the study of gesture (Kendon, 1972; 1980; McClave, 1991) since “their tight connection is a facet of the strong underlying linkage between gesture and speech in general, which in turn is felt to exist because speech is a fundamentally embodied phenomenon” (Loehr, 2014: p. 1388)
The Terminal Breaks (TB) mark the boundaries of the Reference for linguistic analysis (RU), which is to say the minimal pragmatically interpretable linguistic entities, while the Non Terminal Breaks (NTB) mark the boundaries of the information units (IU) that make up the RUs, allowing the identification in the flow of speech of the two hierarchical levels with functional values into which speech is structured, according to this theory
3.1 Gesture and Prosody The gesture annotation system implemented is based on the Linguistic Annotation System for Gestures (LASG) by Bressem et al (2013), in which hand movements are taken as the principal element in the annotation process
Summary
The connection between gesture and prosody is considered fundamental to the study of gesture (Kendon, 1972; 1980; McClave, 1991) since “their tight connection is a facet of the strong underlying linkage between gesture and speech in general, which in turn is felt to exist because speech is a fundamentally embodied phenomenon” (Loehr, 2014: p. 1388). This paper proposes an annotation model for multimodal speech corpora aimed at highlighting the functional and structural correlations between the level of co-verbal gestures and prosody. The present article is aimed in particular at illustrating the annotation schema for gesture and prosody and focuses on the results obtained from the synchronization of gesture / prosody in spontaneous speech.
Sign-in/Register to view highlights & summary Sign-in/Register to access full text options 
Free) 
Talk to us
Join us for a 30 min session where you can share your feedback and ask us any queries you have
Schedule a call
