Multimodal Artifacts Research Articles

Digital Explanation as Assessment in University Science

Assessments in tertiary science subjects typically assess content knowledge, and there is current need to both develop and assess different forms of knowledge and skills, such as communications and digital literacies. A digital explanation is a multimodal artefact created by students to explain science to a specified audience, which is an alternate form of assessment that has potential to develop and assess these other important forms of knowledge and skills. This research draws from perspectives in multimodality, educational semiotics and science education to gain a better understanding of digital explanation as a form of assessment in university science. Data sources include digital artefacts (n = 42), task descriptions and rubrics and pre-/post-interviews (n = 21) with students who created them as a task in a university science subject. Analysis involved identifying the range of media resources used across the data set, seeking patterns in how multiple resources were used and exploring students’ perspectives on the task, including their design decisions. A more detailed look at artefacts from three different science learning contexts illustrates that students base their design decisions on the content knowledge being represented, their technical capabilities to generate them and how to engage the audience. Students enjoy this form of assessment and feel that the tasks allowed them to demonstrate different sorts of capabilities than are normally assessed in their subjects. Recommendations for instructors provide guidance for considering this sort of task in tertiary science contexts.

Saying “story” in the newsroom. Towards a linguistic ethnography of narrative lexicon in broadcast news

Despite a general agreement on the narrative nature of news, the question of what it means for the journalists to tell a story is usually taken for granted, while the analysis of the actual narrative practices in the newsrooms often remains shallow. A way of overcoming this state of affairs is to have a look at the narrative practices and norms in the newsroom. On the one hand, one can track the sites of narrative engagement in the newsroom, where journalists are telling or handling stories in order to achieve their work of making news. On the other hand, one can track the metacommentaries that foreground a narrative orientation to news, when journalists evaluate storying choices or when they use a narrative-related lexicon. This paper explores the latter aspect by tracking the uses of the word « histoire » (story) in the newsroom of a Swiss Public Broadcasting Corporation. The paper identifies and analyses three different meanings of « histoire »: « histoire » as a genre, « histoire » as a set of information and « histoire » as a semiotic product. As a reflexive means, « histoire » enables the media practitioners to navigate the very practical tasks entailed by the production of the multimodal artefact that a television news item is

Multimodality. Foundations, research and analysis. A problem-oriented introduction.

This textbook provides the first foundational introduction to the practice of analysing multimodality, covering the full breadth of media and situations in which multimodality needs to be a concern. Readers learn via use cases how to approach any multimodal situation and to derive their own specifically tailored sets of methods for conducting and evaluating analyses. Extensive references and critical discussion of existing approaches from many disciplines and in each of the multimodal domains addressed are provided. The authors adopt a problem-oriented perspective throughout, showing how an appropriate foundation for understanding multimodality as a phenomenon can be used to derive strong methodological guidance for analysis as well as supporting the adoption and combination of appropriate theoretical tools. Theoretical positions found in the literature are consequently always related back to the purposes of analysis rather than being promoted as valuable in their own right. By these means the book establishes the necessary theoretical foundations to engage productively with today's increasingly complex combinations of multimodal artefacts and performances of all kinds.

“My Story Came to Life!”: How Multimodality Can Inspire Revision in Writing

Revision has been studied mostly from a language centric viewpoint. It is an understudied topic and oftentimes viewed as an afterthought in classrooms rather than a central facet at the heart of quality writing. The focus of this article is on teaching revision via multimodality as a way to serve all students, especially gifted students. This article highlights 27 gifted students’ multimodal artifacts and discussions about the revision process in writing when paired with multimodality. The premise is that teaching revision using multiple modes of communication can lead to deeper, abstract thinking, where students can visualize their writing, thus leading to more revision choices. It is hoped that teachers can routinely use this strategy during writing instruction.

Comicbooks as cultural archeology

The aim of this article is to discuss how comicbooks, as complex multimodal artifacts, discursively interpret and evaluate events during World War II. More specifically, I examine how women are represented in an archive of 69 Captain America stories published by Marvel Comics between March 1941 and March 1943, drawing on theoretical developments within the area of social semiotics (Halliday, 1978; Hodge and Kress, 1988). An important argument within social semiotics and multimodal studies is that texts are a combination of semiotic resources that are used for production and consumption of meanings, in specific contexts (Hodge and Kress, 1988). As a consequence, by examining these semiotic resources one can develop a better understanding of social meanings that circulate in society and how these meanings are structurally organized. By examining forms of discourse for social analysis we may identify ways of addressing particular issues at the level of representation.

Multimodal Artifact Metrics for Conductive Synthetic Resin Products

Multimodal Artifact Metrics for Conductive Synthetic Resin Products

Supporting multimodal literacy in library instruction

PurposeStudents interact with information in many ways throughout the day, code switching between modes depending on their needs. Educators are finally realizing that composing in more than one mode is not only important, but also necessary. The purpose of this study is to examine the role of the academic library, the ACRL Framework and information literacy instruction in creating ethical, inspired users.Design/methodology/approachThis paper looks at previously published work on multimodal discourse, how libraries have supported modes in the past and how the ACRL Information Literacy Framework highlights the need to teach students and faculty how to compose in many modes.FindingsLibrarians are already well-versed in many literacies, including information, visual and media. They are familiar with multimodal tools and the ethical issues related to the use of images, videos and sound files. While professors are proficient in subject matter, librarians are experts in the paradigm shift from print to multiple modes; therefore, by teaching faculty and students to locate, evaluate, use ethically and cite various modes, librarians become the primary resource on campus for creating multimodal artifacts. The strata used by Kress and Van Leeuwen, coupled with the ACRL Framework, are a model for future instructional design.Originality/valueWhile much has been written on visual literacy, little is written on library support of multimodal discourse or combining several modes in one argument. This paper is alone in reviewing the past support of multimodal literacy in libraries and gives some sample activities for use in the academic library.

СТРУКТУРА ПОЛИСЕМИОТИЧЕСКОГО АРТЕФАКТА: МУЛЬТИМОДАЛЬНЫЙ АНАЛИЗ НАЧАЛЬНОЙ ВЕБ-СТРАНИЦЫ ВИДЕОБЛОГ-КАНАЛА НА ХОСТИНГЕ ‘YOUTUBE’

The article is devoted to the analysis of the system of verbal and non-verbal resources and their structuring on the initial web page of a video channel considered as a multimodal artifact, as well as the description of the realization of the meaning-generating potential of elements belonging to different semiotic modes. The authors pay special attention to the cluster analysis of the web page multimodal system, as well as to the phenomenon of performatization and the “multimodal performative”, which not only describes the action, but also has a potential for action - transition to the resource of the user’s interest.

Image-schematic scaffolding in textual and visual artefacts

This paper expands the understanding of how image schemas, while essentially spatial in nature, allow more complex concepts involving non-spatial elements to emerge later. We suggest that the explanation requires adding viewpoint characterization to the concept of image schemas. It is their viewpoint affordances which allow image schemas to form the conceptual scaffolding which becomes subsequently enriched through frames, applied metaphorically, and/or blended with textual and/or visual representations, yielding new and complex meanings in a wide array of multimodal artefacts. As a case in point we study examples instantiating the barrier schema across a wide range of text types (poetry, prose, political discourse) as well as in visual and material artefacts such as cartoons, graffiti or film, showing how people ‘fill in’ the skeletal structure of a barrier, through frames, metaphors and blends, often resulting in a changed embodied interaction with the barrier (characterized by restricted permeability, mobility, vision, or control) or a reconstrual of its materiality, making it (fictively) permeable, transparent, etc. The cross-modality approach we adopt in this research supports the idea that image schemas are not just linguistic (i.e. prompted and maintained through language) but truly conceptual and psychologically real.

Dynamic intersemiosis as a humour-enacting trigger in a tourist video

This article seeks to explore the use of multiple semiotic resources in a multimodal text, namely a YouTube video on the Hans Brinker Budget Hotel in Amsterdam. More specifically, it observes how dynamic intersemiosis is adopted for humorous meaning-making and with an ultimate promotional function. First, it is here claimed that dynamic intersemiosis acts as the humour-enacting trigger enabling script shift, i.e. the semantic process of frame reversal inherent in humour generation. Then, multimodally expressed humour is discussed as performing a parody of similar tourism and eco-tourism texts and of the specialized language they adopt. The questions raised are: How do dynamic images and the soundscape combine in terms of congruence and dissonance? How does intersemiosis act as a humour-enacting trigger in the multimodal artefact? How does multimodally expressed humour position the Hans Brinker Hotel within a tourism discourse? How and why is the parody of promotional (eco)-tourism videos realized?

Viewpoint and image schemas in multimodal political discourse

This paper investigates the ways in which multimodal artefacts on social media can infuse a simple object with political meaning. Using a cognitive linguistic framework, I examine the sudden conceptual enrichment of the umbrella, an object that became the defining symbol of the 2014 Hong Kong suffrage protests. The paper contends that the enrichment of the umbrella, which allowed it to represent the political values of the protesters, was enabled by the conceptual integration of cultural frames and viewpoints in artistic illustrations, which retained the shield-like functionality of the umbrella while simultaneously enhancing its symbolic meaning. First I explore the material affordances of the umbrella and the rudimentary schemas that restrict how the umbrella can be depicted. Second, I analyze two popular artefacts disseminated online. One is a cartoon that draws from the iconic “Tank Man” photo from Tiananmen Square. The other artistically alters the Hong Kong flag to look like a configuration of umbrellas. Building on the basic schemas of the umbrella, both multimodal artefacts insert new viewpoints that prompt the re-construal of the original frame. Third, I posit that the new frames and schemas of the artefacts informed how protesters and the Hong Kong government understood the movement as it unfolded.

A Multiliteracies Approach to Teaching Korean Multimodal (Im)politeness

ABSTRACTThis study provides and tests pedagogical materials for teaching Korean multimodal (im)politeness, reflecting recent findings on the role of nonverbal means of communication in the communication of (im)politeness. A pedagogy of multiliteracies is implemented in this study, which supports the development of language learners' multimodal competence. Within the multiliteracies classroom, learners are encouraged to actively express their identities via language and other multimodal means. In order to examine the effectiveness of multiliteracies framework in enhancing identity negotiation, the activities performed by language learners to express their identities are analyzed from an activity-theoretic perspective. This study includes an 80-minute lesson designed around the theme of yeyuy ‘manners, etiquette’ in Korea for intermediate-level Korean learners as well as a qualitative analysis of the classroom interaction where the lesson was carried out. The analysis shows that language learners construct and negotiate their first-language and second-language identities producing multimodal identity artifacts, while expressing their own perspectives on Korean (im)politeness and dealing with the views held by the teacher, their classmates, and materials. It is also demonstrated that learners recognize the significance of multimodal resources involved in authentic media texts and critically evaluate the (im)politeness-related meanings that the resources intend to signal.

An overview of research within the Genre and Multimodality framework

This review article provides an overview of the research conducted within the Genre and Multimodality framework, which has been used to describe the multimodality of page-based documents and other multimodal artefacts over the past 15 years. The article explicates the motivation and inspiration for developing the framework, introduces its central theoretical concepts and presents its applications across a number of case studies. Finally, the article discusses the criticism directed towards the model and identifies avenues of future development.

マルチモーダル人工物メトリクスの研究（合成樹脂製品への適用）

マルチモーダル人工物メトリクスの研究（合成樹脂製品への適用）

Reflexive language and ethnic minority activism in Hong Kong

This article engages with Archer’s call to further research on reflexivity and social change under conditions of late modernity (2007, 2010, 2012) from the perspective of existing work on reflexive discourse in the language disciplines (Silverstein 1976, Lucy 1993). Drawing from a linguistic ethnography of the networked trajectories of a group of working-class South Asian youth in Hong Kong (Pérez-Milans & Soto 2014), we analyze the trajectory of Sita, a Hong Kong-born young female with Nepali background. In her trajectory, performative acts of ethnic minority-based activism emerge as key in the enactment of a given set of values, stances, types of persona and situated forms of alignment/disalignment. That is to say, Sita’s enactment of activism is seen in this article as tied to a discourse register (Agha 2007: 147). As such, ‘talking/doing activism’ is inter-textually linked to a speech chain network of a group of secondary school students, teachers, researchers and community-based minority activists engaged with Sita in various interrelated projects for social empowerment. Analysis of interview transcripts, online chats and multimodal artifacts shows the extent to which the coordinated formation of this discourse register proved useful in providing Sita with relevant cultural capital (Bourdieu 1986) with which she shaped her own academic trajectory, from a low-prestige government-subsidized secondary school to an elite international college. Data also point towards the need for further engagement with recent invitations to re-imagining identity and social action under current conditions of diversification (Blommaert 2013).

A Study of Cohesion in International Postgraduate Studentss Multimodal Management Accounting Texts

This study investigates the key multimodal academic literacy and numeracy practices of five international postgraduate students enrolled at an Australian university. Specifically, it aims to provide an account of the salient textual and the logical patterns through the analysis of cohesive devices in a key topic in the Management Accounting course, namely budgeting schedules. The research study employs a Systemic Functional Multimodal Discourse Analysis (SF-MDA) of texts (Alyousef, 2013, 2015; Alyousef & Mickan, forthcoming). This approach is framed by Halliday’s (1985) Systemic Functional Linguistics (SFL) approach and Halliday & Hasan’s (1976) cohesion analysis schemes. SFL provides powerful analytical tools for foregrounding the processes through which students construct disciplinary specific knowledge in a community through academic literacies. Lexical cohesion formed the largest percentage of use, and in particular repetition of the same lexical items, followed by reference.The findings contribute to the description of the meaning-making processes in these multimodal artifacts. They provide a potential research tool for similar multimodal investigations across a broad range of educational settings. Implications of the findings are finally presented.

Refugee-themed picture books for ethical understanding in curriculum English

ABSTRACTThis study looked at the curricular resource potential of refugee-themed picture books for embedding an ethics of responsibility for linguistic diversity into the subject of English studied by all students in English-dominant western societies. Selected picture books were analysed in terms of a Levinasean ethics of responsibility for alterity in order to probe the possibilities of a supplement or alternative to the ethics of empathy long established in the literary component of English. The analyses looked at the books as multimodal artefacts, understanding the linguistic modes of the texts from a translingual perspective. This perspective construes language as mobile semiotic resources that are rendered meaningful through the performances of interlocutors in a given communicative situation. From the analyses, findings were produced about (1) representations of the monolingual sameness of everyday life in Australia as an English-dominant society and (2) the translingual dispositions called forth when language variation enters into the linguistic text. It is concluded that the ethics of responsibility for alterity holds out promise for tapping the curricular resource potential of refugee-themed picture books for embedding ethics into the language awareness component sometimes incorporated into English.

A study of cohesion in international postgraduate Business students’ multimodal written texts: an SF-MDA of a key topic in finance

Empirical research studies of finance students’ language use have investigated students’ performance in finance courses and the effect of class attendance on students’ performance.Similarly, research on accounting students’ texts has been directed at readability of accounting narratives and lexical choices. Systemic Functional Linguistics (SFL) based research in multimodal communication and representation has been confined to school and workplace contexts. Whereas multimodal communication investigations in tertiary contexts has been conducted across the fields of mathematics, science and computing, and nursing, business courses have not been explored. The purpose of this paper is to report on a case study designed to investigate the key multimodal academic literacy and numeracy practices of ten international Master of Commerce Accounting students enrolled at an Australian university. Specifically, it aims to provide an account of the salient textual and the logical patterns through the analysis of cohesive devices in a key topic in the Principles of Finance course, namely capital budgeting techniques and management reports. This study is pertinent as most international ESL/EFL students’ enrolments in Australia and elsewhere is in business programs. This study is underpinned by Halliday’s (1985) Systemic Functional Linguistics (SFL) approach to language and Halliday and Hasan’s (1976) cohesion analysis scheme. The study employs a Systemic Functional Multimodal Discourse Analysis (SF-MDA) for the analysis of cohesive devices in the participants’ multimodal texts. Lexical cohesion formed the largest percentage of use, and in particular repetition of the same lexical items, followed by reference.The findings contribute to the description of the meaning-making processes in these multimodal artefacts. They provide a potential research tool for similar investigations across a broad range of educational settings. Implications of the findings for finance students and educators are finally presented.

A study of Cohesion in International Postgraduate Students’ Multimodal Management Accounting Texts

This study investigates the key multimodal academic literacy and numeracy practices of five international postgraduate students enrolled at an Australian university. Specifically, it aims to provide an account of the salient textual and the logical patterns through the analysis of cohesive devices in a key topic in the Management Accounting course, namely budgeting schedules. The research study employs a Systemic Functional Multimodal Discourse Analysis (SF-MDA) of texts (Alyousef, 2013, 2015; Alyousef & Mickan, forthcoming). This approach is framed by Halliday‟s (1985) Systemic Functional Linguistics (SFL) approach and Halliday & Hasan‟s (1976) cohesion analysis schemes. SFL provides powerful analytical tools for foregrounding the processes through which students construct disciplinary specific knowledge in a community through academic literacies. Lexical cohesion formed the largest percentage of use, and in particular repetition of the same lexical items, followed by reference.The findings contribute to the description of the meaning-making processes in these multimodal artefacts. They provide a potential research tool for similar multimodal investigations across a broad range of educational settings. Implications of the findings are finally presented.

Multimodal Self-Authoring Across Bi/Multilingual Educator and Student Learning Spaces

Drawing on sociocultural perspectives, this article illustrates the potential for extending figured worlds through cultural artifacts. Using thematic and multimodal analysis, the authors first compare how spaces of authoring were produced in two professional development settings: Proyecto Bilingüe and Critical Education Studies. Following one educator from each of these settings into a classroom and into digital spaces respectively, the authors then compare the multimodal cultural artifacts—journey boxes and digital iJourneys—produced by their bi/multilingual students. The findings show the importance of producing spaces of authoring for identity negotiations and the role multimodal cultural artifacts play in embracing memories and lived experiences across figured worlds for both students and educators. Implications for teacher educators and bilingual teachers are discussed.