Analysing Multimodal Texts in Science—a Social Semiotic Perspective

Abstract

AbstractTeaching and learning in science disciplines are dependent on multimodal communication. Earlier research implies that students may be challenged when trying to interpret and use different semiotic resources. There have been calls for extensive frameworks that enable analysis of multimodal texts in science education. In this study, we combine analytical tools deriving from social semiotics, including systemic functional linguistics (SFL), where the ideational, interpersonal, and textual metafunctions are central. In regard to other modes than writing—and to analyse how textual resources are combined—we build on aspects highlighted in research on multimodality. The aim of this study is to uncover how such a framework can provide researchers and teachers with insights into the ways in which various aspects of the content in multimodal texts are communicated through different semiotic resources. Furthermore, we aim to explore how different text resources interact and, finally, how the students, or authors of teaching resources, position themselves in relation to the subject. Data consist of one student text and one teaching resource text, both comprising drawn and written elements in combination with symbols. Our analyses of the student text suggest that the proposed framework can provide insights into students’ content knowledge and, hence, how construction of multimodal texts may be a useful tool for formative assessment. When it comes to teaching resources, the framework may be a useful tool for teachers when choosing resources, particularly in relation to students’ possibilities of meaning making when engaging with such texts, but also, as a basis for classroom discussions.