Systemic Functional Multimodal Discourse Analysis Research Articles

ANALYSIS OF TEXT-IMAGE RELATIONSHIP IN SELECTED ONLINE NIGERIAN NEWSPAPER CARTOONS ON COVID 19 PANDEMIC

The study conducted an analysis of text-image relationship in selected on-line Nigeria Newspaper cartoons, using Systemic Functional Multimodal Discourse Analysis (SF-MDA) developed by (Djonov, 2005 and O’Halloran, 2008) as approach to the COVID 19 cartoons. A descriptive and discursive approach were adopted for the study. The corpus of the study consisted 6 (Six) online COVID-19 cartoons which were drawn from the following 3 (Three) online National Newspapers in Nigeria: The Punch, The Iniquere and The Niche, published from the period of March, 2020 and December, 2020. The findings of the study revealed that, COVID 19 cartoons rely heavily on the interplay between images and words specific language use especially caricature as a tool to communicate social and political messages. Thus, through creative combination of images and words, COVID 19 cartoons have the potentials of entertaining and informing the public on current issues. The study therefore recommended that, the use of irony by using words or pictures to describe the opposite of what is such as what should be in COVID 19 cartoons should also be investigated.

Revealing the Beauty Standard of Miss Universe 2019 in Video Advertisement: Systemic Functional Linguistic

This study examines the YouC 1000 'Confidence' edition video advertisement, focusing on the interplay between verbal and visual elements to convey a nuanced message about beauty, health, and cultural identity. The purpose of this study is to reveals the interplay between verbal and visual that constructing beauty standard meaning of Miss Universe 2019. Qualitative study as the method applied and documentation as the data. There are several theories implies to find out the beauty standard including Systemic Functional Multimodal Discourse Analysis (SF-MDA) by O’Halloran (2004), Systemic Functional Linguistic especially in Transitivity and Mood analysis by Halliday (1960), Visual Grammar especially in Representational meaning by Kress and Leeuwen (2006), and Interplay between verbal and visual. The findings shows that verbal and visual elements work together to create meaning in beauty and convey that beauty is exist in confidence, well-being, and inner validation.

How Audience Reaction to The 5th Estate Vlogs Aid in The Construction of Mutahi Ngunyi’s Identity as an Authoritative Political Analyst

This study investigated one of Kenya’s foremost political commentary vlogs – The 5th Estate as a rich site on how multimodal resources are marshaled by Mutahi Ngunyi, the vlog’s creator and political analyst, to construct the identity of an authoritative political analyst. Whereas there are several studies on identity construction on social media platforms, little research on identity construction focusing specifically on vlogging exist. More so, there is scanty investigation of identity construction strategies used by vloggers who specialise in political commentary. Using the Systemic Functional – Multimodal Discourse Analysis (SF-MDA) approach, the research investigated how audience reaction aid in the construction of individual’s identity. The research adopted a qualitative perspective and more specifically a descriptive-interpretive design. Data which included vlogs posted on The 5th Estate since its establishment on 19th February, 2019 to date constituted the population of study from which four most popular vlogs were sampled. Data for research was collected, transcribed and analysed using content analysis technique. Knowledge generated by this study is expected to fill the existing gaps in political communications and new media research concerning identity construction in social media. The study could also be useful to politicians, political influencers and government agents who may want to adopt vlogging to communicate to their audience efficiently and effectively. The study concluded that audience reaction plays an important role in shaping Identity of an individual by affirmation of the message, use of descriptive words that describe the author, or titles ascribed to the author.

The Representation of Counterproductive Religious Values in a Selected Chapter of an Indonesian ELT Textbook: Systemic Functional Multimodal Discourse Analysis

Countless studies have examined the vital role of ELT textbooks as learning sources, particularly in terms of intercultural, multicultural, and trans-cultural analysis. Yet, none of them specifically talked about religious values as the research focus. Hence, this study aimed at construing religious values represented in a selected chapter of an Indonesian ELT textbook. Descriptive problem-driven content analysis was used as the research design, whilst the research data were collected through document analysis. Later, to analyze the data, the research utilized Systemic Functional Multimodal Discourse Analysis (SF-MDA) (O’Halloran, 2008c) as the framework with a focus on representational meaning and transitivity analysis for each visual and textual data. The findings showed that two data modes represent religiosity, namely visual and verbal data. In visual, religious values (artifacts, beliefs, and behaviors) are represented by the classificational process while in verbal data; they are represented by the material and relational processes. Four of Indonesia's large recognized religious communities were represented namely Christianity, Buddhism, Hinduism, and Confucianism. Yet, there is no single datum that figures Islamic values, whereas, the Islamic community is the largest in the country and even in the world. Hence, this implication suggests that stakeholders (particularly textbook authors) should pay attention to the issue of how to fairly present the five legalized communities' values existing in Indonesia. Therefore, because Indonesia has varied its communities, ethnicities, and backgrounds, ELT textbooks should fairly embody the diversities more over the religious aspects which are the core competence to gain.

The Functional Grammar of Dance applied to ELAN annotation: meaning beyond the naked eye

Abstract The main purpose of this article is to show how Functional Grammar of Dance (FGD) can be applied to dance discourse analysis to reveal the discursive strategies that underlie choreographies and support movement-based communication. In recent years, studies in Systemic Functional Multimodal Discourse Analysis (SF-MDA) have benefited from the development of several models of analysis, and from the emergence of a new focus on the materiality of semiotic modes. The Functional Grammar of Dance is a recently developed model of analysis that draws on Systemic Functional Linguistics theory and focuses on movement-based communication starting from dance (particularly ballet as a starting point). The Functional Grammar of Dance has been adopted as a central theory in an international collaborative project funded by the AHRC (Arts and Humanities Research Council) in the UK and the DFG (Deutsche Forschungsgemeinschaft, ‘German Research Foundation’) in Germany, The Kinesemiotic Body, where it has been used with the ELAN annotation system. ELAN annotation based on the Functional Grammar of Dance was used for both live-collected data from dance rehearsals and for dance videos and revealed aspects of dance performances that are not necessarily captured by the naked eye. This paper proposes three case studies where we show that our ELAN annotation with the FGD provides an effective and very flexible interface for the grammaticalization of movement-based communication and the study of movement-based discursive strategies.

A new way to justice? Multimodal meaning construction in videoconferencing technology-based online criminal courts in China

ABSTRACT During the COVID-19 pandemic, the Chinese state temporarily locked down physical courtrooms and fully relied on online trials based on videoconference technology. This raises the question of how computer-mediated communication affects the realization of justice. This question is particularly significant for criminal cases, where assessing the credibility of evidence, debating between involved actors, and establishing legal identity heavily affect the life-altering sentence. Based on the systemic functional-multimodal discourse analysis (SF-MDA) of a case recording, this study shows that China’s online courts are textually unregulated, with inscribed unbalanced power relations that favor the judge and/or procurator against the defendants. It also finds ineffective ideational meaning-making, the failure to clarify legal jargon and procedures for defendants, and the lack of defense support. These problems illustrate that the current adoption of videoconferencing technology in criminal trials can seriously undermine the court’s constitutionality and legitimacy. Although online courts are a welcome alternative that can help protect public health and avoid delays in the disposal of cases, their currently disorganized implementation, affordabilities, and technical problems compromise justice.

The Conceptualisation of User-App Interactivity in Augmented Reality-Mediated Learning: Implications for Literacy Education

Augmented reality (AR) is transforming users’ multisensory experiences and heightening the level of engagement with multimodal learning. Scholarly attention is urgently needed to conceptualise and examine user–app interactivity in educational contexts. Drawing on the systemic functional–multimodal discourse analysis approach, this article aims to explore key user roles prompted by AR apps and examine educational functions that these user roles fulfil in AR-mediated learning. Based on our analysis of 14 AR apps selected for a 3-day workshop with six Australian primary school teachers, we identified four categories of user roles that facilitated different literacy activity types during AR-mediated learning. To design effective learning experiences, this article argues that teachers need to consider the resonance between students’ AR experiences and their prior engagement with other forms of digital texts when planning for scaffolding strategies.

DECODING SOCIO-COGNITIVE ELEMENTS IN INDONESIAN ONLINE NEWS: A SYSTEMIC FUNCTIONAL MULTIMODAL DISCOURSE ANALYSIS

This study explores the representation of socio-cognitive components in Indonesian political multimodal news text using the framework of Systemic Functional Multimodal Discourse Analysis (SF-MDA). With the increasing prominence of multimedia in contemporary news production, understanding how socio-cognitive aspects are conveyed through multiple modes becomes essential. Drawing on SF-MDA, which combines systemic functional linguistics and multimodal analysis, the study examines how Indonesian political news texts incorporate social and cognitive elements to shape meaning and influence audience perceptions. This study was undertaken qualitatively involving three online news published by Indonesia’s publishers as the data. The result reveals that the visual and verbal modes merged in the news tend to highlight the good image of the political figures. It means that the meanings of visual and verbal components of the news persuasively work together to create the good impressions among readers to the human actor in the news. As the implication, multimodal resources are strategically employed to shape public opinion and promote specific political ideologies. Ultimately, this research aims to contribute to the development of critical media literacy and foster a more nuanced understanding of the socio-political dynamics in contemporary Indonesia.

Deployment of gestures in the semiotic construction of scientific knowledge: a systemic functional approach to pedagogic semiosis

Abstract A variety of semiotic resources makes the construction of scientific knowledge possible and meaning-making resources are conveyed by certain semiotic modes. Next, numerous studies have demonstrated the pedagogical importance of gestures in the demonstration of scientific knowledge in the classroom. Drawing on social semiotic systemic functional theory and legitimation code theory, this study explores the types and the role of gestures in the semiotic construction of scientific knowledge in pedagogic semiosis and their pedagogical values for the meaning-making of science content. With this aim, a gesture observation protocol (GOP) is developed to annotate and analyze data. Two science lectures, which are broadcast remotely for middle school students including gifted students, are observed. Systemic Functional Multimodal Discourse Analysis (SF-MDA) method is employed to analyze and evaluate data. Results demonstrated that the deployment of gestures that contributes to the construction of scientific knowledge is low and deployed gestures mostly do not have pedagogical value. It is recommended that teachers should be more conscious and educated about deploying gestures to present scientific knowledge efficiently.

The Ideational Meanings of Multimodal Texts Presented in an EFL Textbook

This study explores the ideational meanings of multimodal texts in an EFL textbook entitled “Bahasa Inggris: When English Rings a Bell”. This study aims to describe the meanings of visual and verbal modes from ideational aspects. The visual modes were analyzed by using Visual Grammar Design framework (Kress van Leeuwen, 2006). Meanwhile, the verbal modes were analyzed using Systemic Functional Grammar theory focusing on the Transitivity system (Halliday, 1994). The ideational aspects of this study focus on components like participants, processes and circumstances. This study applied Systemic Functional Multimodal Discourse Analysis (henceforth SF-MDA) methodology. The findings of this study found that the dominant process types used in realizing the ideational meaning of the verbal text is relational process. Meanwhile, the ideational meaning of visual text is realized mostly through speech process. This study showed that the presentation of multimodal texts in the EFL textbook is relevant to the teaching and learning activity for the Junior high school students. Keywords: multimodality, ideational meaning, systemic functional grammar, multimodal analysis

Designing Knowledge Dissemination in a Digital Era – Analysing TED Talk’s Multimodal Orchestration

Online learning has gained increasing attention during the COVID-19 pandemic. Teachers face social exigencies to design ways of knowledge dissemination in online instruction. We posit that understanding how knowledge can be represented in successful online academic genres can inform teachers on how they can design students’ online learning experiences. This study examined how scientific knowledge is disseminated in one of the most widespread academic genres, TED Talks, which share discoursal similarities with other academic genres such as online lectures. This study adopted a systemic functional multimodal discourse analysis approach to explore how a presenter used speech, images, and gestures to disseminate knowledge. The analysis shows that a presenter orchestrates speech, images, and gestures strategically to clarify the scientific ideas and engage the audience. Based on understanding how the three semiotic modes are used to disseminate scientific knowledge in accessible and engaging ways, this paper discusses how insights on multimodal orchestration can function as a heuristic tool to inform design in online learning.

Introducing a multimodal perspective to emotional variables in second language acquisition education: Systemic functional multimodal discourse analysis.

A systemic functional theory applied to multimodal discourse analysis (SF-MDA) is related to the theoretical and practical meaning extraction from the data showing how multiple semiotic resources are used in different modes of discourse ranging from textual, printed, and electronic texts to visual language and those existing alive as embedded in reality. In the present paper, we provide an introduction to the SF-MDA approach and then justify how it can benefit the second language acquisition (SLA) domain and more specifically the exploration of L2 affective variables. We will show how the SF-MDA approach fits in with the complexity and dynamic systems theory (CDST), and is capable of revealing the nuances of the developmental nature of the affective variables involved in language learning. The exemplary works of research in the SLA domain using the SF-MDA approach are reviewed here as well as those exploring L2 affective variables. The further benefits of this approach will be emphasized at the end along with the implications it can have for the CDST-led line of research in applied linguistics.

Systemic functional multimodal discourse analysis: multimodal composing and civic agency of multilingual youth

ABSTRACT In our civic engagement afterschool program in the southeast of the United States, our youth and adult participants used a wide range of modalities including mapping, rapping, drawing, and performance to construct and convey their unique visions of school and community spaces. The purpose of this methodological paper is to explore systemic functional multimodal discourse analysis (SF MDA) as a potential resource to gauge the effectiveness of such humanizing and multimodal spaces in positioning youth as civic leaders. Data collected for this study included curriculum materials, descriptive field notes, and video recordings of the multimodal processes of focal youth across five program modules. Informed by an SF MDA perspective, our multimodal transcriptions illustrate how youth participants engaged in an intersemiotic complementarity of modalities and languages to share their insights and enact civic identities.

‘IS THE PICTURE WORTH A THOUSAND WORDS?’: THE INTERPERSONAL MEANINGS OF A DIALOGUE IN AN EFL TEXTBOOK

This study was aimed at scrutinising a multimodal text embedded in a dialogue of an EFL textbook. To this end, Systemic Functional Multimodal Discourse Analysis (SF-MDA) focused on interpersonal meanings consulting grammar of visual design and intersemiotic complementarity drawing on systemic functional linguistics were employed to analyse the artefact, a dialogue within a part named 'communication' taken from an EFL textbook for a primary school level. The findings revealed that declarative clause and modalisation (epistemic modality) of probability was found to be the most common system used in the verbal text. Meanwhile, high modality and validity were found in the visual image indicated by the realisations and representations of detailed abstraction and full-colour saturation. Finally, the study draws a conclusion that there is a cohesive interaction to a certain extent between the verbal text and the visual image represented in the multimodal dialogue.

Multimodal knowledge building in a Japanese secondary English as a foreign language class

Multimodal analysis examines how different modes, such as space, gesture, and language, instantiate meaning together. In this paper, a Systemic Functional-Multimodal Discourse Analysis demonstrates how teachers enact their pedagogy with their students across modes through what is represented experientially, how relationships between people are construed interpersonally, and how coherent texts are realized textually. This paper is a preliminary study of classroom data from a larger project looking at the multimodal pedagogy of Japanese secondary school teachers of English through the paired lenses of Systemic Functional-Multimodal Discourse Analysis and Legitimation Code Theory. It demonstrates how methods from these perspectives may be productively combined. How this teacher builds cumulative knowledge multimodally can be uncovered through the analysis of pedagogic register (Rose, 2018) and exchange (Berry, 1981; Martin and Rose, 2007), as well as classroom space and representing and textual action (Amundrud, 2017; Martin and Zappavigna, 2019). How both gesture and dialogic exchange between the teacher and students modulate the contextual relation of the knowledge construed in class is also explored via semantic gravity, which looks at how closely connected knowledge practices are to their context (Maton, 2014). As a preliminary study, the paper closes with limitations and future directions for this pedagogic multimodality research.

Managing English Young Learners’ Classroom Activities through Gestures: A Multimodal Perspective

Managing classroom activities in children has become a challenge for English as a Foreign Language (EFL) teachers. Teachers usually use gestures in class activities, such as giving directions and pointing at students. This study explores the multimodal representation of the teacher's gestures when managing classroom activities in the EYL context. Data were collected through non-participant observation and analyzed by Systemic Functional Multimodal Discourse Analysis (SFMDA). The findings show that four meanings are multimodally represented in teacher gestures, namely (1) building student-teacher relationships through clapping and thumbs up, (2) emphasizing instruction through raising hands, pointing and lifting class objects, (3) encouraging involvement of students in academic tasks through finger pointing and counting down, and (4) warning of disruptive behavior of students through pulling gestures. Pedagogically, this research provides a paradigm shift that classroom-based communication does not have to use a single semiotic source but also a combination of other semiotic sources to help students understand teacher instructions easily.

‘Can We See It?’: Contextualizing ‘Deforestation’ from an English-Medium Science Textbook for a Primary School Level

Understanding an English-medium science textbook is possibly challenging for some students. It is, for example, due to the language used. To deal with this issue, construing the use of the other mode, such as visual images, along with the verbal text is regarded useful. Thereby, the construal of multimodality in an English-medium science textbook becomes crucial. Albeit a myriad of inspections on multimodality exists, but to the best of the writer’s knowledge, such investigation with respect to an English-medium science textbook, particularly at a primary school level, was found to be limited. Therefore, this study aimed to scrutinize the verbal text and visual image presented in a science textbook used for a primary school level which is presented in English. To that end, a descriptive research design was employed. In this regard, a systemic functional multimodal discourse analysis (SF-MDA) within the trinocular metafunctions encompassing ideational, interpersonal, and textual metafunctions was utilized. The systemic functional linguistics theory, the grammar of visual design, intersemiotic complementarity, and logico-semantics were the frameworks employed to analyze the artefact, the English-medium science textbook. The findings revealed that the visual image and verbal text interact with one another within the three metafunctions. Given the interaction between the two modes, the present study suggests that both teachers and students are required to take into considerations and be aware of the potential or roles of images along with the verbal text, i.e. the images are not merely accessories, but instead, these are able to assist the comprehension of the science materials learned.

Interpersonal Meaning of Systemic Functional Multimodal Discourse Analysis on World Immunization Week 2021 Posters Related to Covid-19 Issue

The purpose of this study is to determine how World Health Organization (WHO) packaged information related to Covid-19 issue, especially in the campaign of vaccination program all over the world. The interactional meaning represented in World Immunization Week 2021 Posters used by WHO in the campaign is examined and analyzed. The analysis focuses on the representation of visual and verbal mode through a Systemic Functional Multimodal Discourse Analysis (SFMDA) approach; this approach makes it possible to complete the analysis by using not only visual grammar in determining the visual mode but also Systemic Functional Grammar in determining the verbal mode. By visual grammar, interactional meaning is analyzed through the concept of interactive meaning involving attitude, social distance, contact, and modality in both interactive and represented participants while by Systemic Functional Grammar, the meaning is analyzed through the concept of interpersonal meaning involving the analysis of mood system. The results reveal that a compact combination between visual mode and verbal mode may increase the viewers’ readability and appropriate knowledge to the message of the posters, and finally, the goals which are expected by WHO as the poster maker in the campaign can be communicated and understood well

A Multimodal Discourse Analysis of Incognegro (2008)

This study attempts to conduct a multimodal discourse analysis (MDA) of Incognegro (2008), a graphic novel by Mat Johnson and arts by Warren Pleece, by applying Michael Halliday’s theory of the Systemic Functional Grammar (SFG) (1994) for the written texts, i.e. the captions found on the images, and Gunther Kress and Theo van Leeuwen’s Grammar of Visual Design (GVD), or what has been recently called Systemic Functional Multimodal Discourse Analysis (SF-MDA) (1996) for the images themselves. The study employs, as well, Teun A. van Dijk’s modal of Critical Discourse Analysis (CDA) (2004), in which power, racism, segregation, oppression, ethnicity, inequality, discrimination, identity, superiority, inferiority, dominant groups, and dominated groups are being analyzed. The study aims at showing the inequality, the oppression, the racial discrimination, and the exercised power Negroes previously suffered (1930s) in America, the land of freedom, and how this suffering is depicted through graphic novels for historical documentation. The study shows that the Whites considered themselves the dominant group, whereas the Negroes were treated as slaves, not even equal to human beings, and hence are recognized to be the oppressed and the dominated group.

A Systemic Functional Multimodal Discourse Analysis of English Micro-lectures

Lim (2019) states that one challenge for the researchers in multimodal discourse analysis is to describe and discuss the interplay across various semiotic resources. English micro-lectures, as a kind of popular and widespread teaching materials in the information age, are typical multimodal discourses involving multi-semiotic resources. This article adopts the systemic-functional synthetic framework for multimodal discourse analysis from Zhang Delu (2018) to explore the relationship of various modes involved in excellent English micro-lectures and further examine how the semiotic resources cooperate and interact to construct communicative meaning. By analyzing and interpreting the context of culture, the meaning, the lexico-grammar, the media and the substance systems of involved modes in micro-lectures, it proves that English micro-lectures demonstrate complicated intersemiosis and various modes cooperate in a perfect way in meaning construction. This comprehensive investigation of semiotic systems sheds light on teachers’ mode choice and teaching design in producing micro-lectures and students’ learning strategies of micro-lectures in the mobile-assisted learning environment.