Multimodal Synthesis and the Voice of the Multimedia Author in a Japanese EFL Context

Abstract

This paper examines a powerful potential of multimodal design: meaning that transcends the total semiotic contribution of a text's constituent parts. With reference to data drawn from the digital storytelling practices of Japanese university students, the author argues and demonstrates that in the current semiotic climate, characterised by the increasing availability and complexity of communication tools and ready appropriation of available designs, practices of multimedia authorship truly can evince expression that is authentically multiplicative. However, this sort of meaning making does not automatically come about. Controlling the inherent polysemy of multimodal texts, in the author's view, is a matter of recognising points of semantic correspondence among co-deployed images, language, etc. and creating syntheses of potential meaning that cut across these semiotic modes. The author further argues that it is in this way that the voice of the multimodal author can most clearly be heard, particularly in cases in which a language learner–author integrates elements within a multimedia text that encode meaning in the L2.