Gesture in multimodal language learner interaction via videoconferencing on mobile devices

Abstract

This thesis focuses on how adult English language learners exploit and experience gesture while communicating with one another via mobile technologies. Mobiles create opportunities for multimodal language learning beyond the classroom (Kukulska-Hulme et al., 2017), however, modes such as gesture are mediated and transformed by technology in complex ways (Hampel & Stickler, 2012). In a small-scale qualitative study, learners from a range of nationalities who were studying on language programmes in the UK were connected in dyads via Skype videoconferencing (VC) in order to complete information gap tasks using tablets, 2-in-1 devices, and smartphones. These communicative tasks had been intentionally designed around a diversity of informal ‘settings’ (Benson, 2011) which included cafes, museums, and historical buildings. Following the tasks, participants took part in stimulated recall interviews in order to reflect on their multimodal forms of communication. This exploratory, qualitative study examines gesture from a theoretical perspective which links the mode to spoken language (Kendon, 2004; McNeill, 1992; Norris, 2004) and positions gesture within the wider framework of the negotiation of meaning (Varonis & Gass, 1985). As the role of speech-associated gestures within language learning via technology has not been widely researched, an interdisciplinary methodology had to be designed to analyse the video recorded data from the learners’ tasks. This is based on transcription procedures from gesture-speech analysis (McNeill, 1992; McNeill & Duncan, 2000). As gesture in this study is understood as being closely aligned to speech, a multimodal unit of analysis was combined with the Varonis and Gass (1985) framework of the negotiation of meaning. The multimodal method allowed for the categorisation and analysis of gesture to investigate how learners may co-orchestrate the two modes in relationship to their deployment of mobile technologies from beyond the classroom. The participants were asked to reflect on their interactions from multimodal perspectives and interview data were triangulated with the task performances. Theoretical and pedagogical conclusions are drawn as to the manner in which learners exploit gesture as an integral part of the negotiation of meaning.