Tracking Discourse Topics in Co-speech Gesture

Abstract

AbstractThis paper argues for the integration of co-speech gesture into formal models of discourse structures. In particular, I use a variation of the Question Under Discussion (QUD) framework to show how features of co-speech gesture may be used to inform discourse analysis. To do this, I look at examples of three discourse relations: clarification, citation and attitude expression. All three of these involve a momentary digression from the main discourse topic, a phenomenon which I call ‘micro-excursion’. I show that these micro-excursions correspond to particular and recurrent changes in co-speech gesture sequences. I provide micro-analyses of three discourse fragments taken from interviews on late night television. For each fragment, I look at speech-gesture alignment and then use this alignment to inform an analysis of the discourse’s structure. In particular, I show the ways in which hand shape and orientation can be used to segment a discourse into different topics. By using features of co-speech gesture in this way, we are able to inform and justify a proposed discourse structure analyses in a principled and non-circular way. Furthermore, I show that this methodology can be straightforwardly integrated into existing models of discourse structure, despite them being designed for mono-modal discourses.KeywordsCo-speech gestureDiscourse structureQuestion under discussion