A multimodal approach to examining 'embodiment' in tangible learning environments

Abstract

Tangible and multitouch technologies offer new opportunities for physically interacting with objects and digital representations, foregrounding the role of the body in interaction and learning. Developing effective methodologies for examining real time cognition and action, and the relationship between embodiment, interaction and learning is challenging. This paper draws on multimodality, with its emphasis on examining the use of multiple semiotic resources for meaning making, to examine the differential use of semiotic resources by pairs of students interacting with a tangible learning environment. Specifically the analysis details the role of body position, gaze, manipulation and speech in shaping interaction. The analysis illustrates the interaction between these modes and 'multimodal action flow', particularly in terms of pace, rhythm and interaction structure, and the implications of this for interaction and the meaning making process.