Creativity and power: A systemic functional multimodal discourse analysis of the co-constructed multimodal creativity-power relation in House M.D.

Systemic Functional Multimodal Discourse Analysis (SF‐MDA) is an extension of the Systemic Functional Theory (SFT) developed by Michael Halliday. This article presents the distinctiveness of SF‐MDA as an approach to multimodality. The characteristics of SF‐MDA, as compared to other approaches to multimodality, can be defined by its adherence to specific principles from SFT and its attention to particular aspects in multimodal analysis. SF‐MDA is characterized by its organization of meaning in relation to metafunctions, the perspective of meanings as made through semiotic choices in a system, systematic fine‐grained analysis, attention to “grammatics,” and interest in intersemiosis. The article reflects on how SF‐MDA has evolved over time, with researchers exploring the intersemiosis between various semiotic resources, such as text and image and text and gesture. It concludes by calling for further research to advance the theory and methodology of SF‐MDA, aiming to account for the full range of semiotic resources present in multimodal texts.

ABSTRACTClassroom research in the past has mostly been based on the study of the language used by the teacher and students. With the availability of video recording technologies, it is now possible to collect multimodal classroom data, such as the teachers’ use of gestures, positioning, and classroom space to enact specific pedagogy. The challenge for the researchers is how to annotate and analyse such multimodal data. This paper proposes an approach for the annotation and analysis of the teachers’ use of gestures in the classroom for meaning-making. Situated within Systemic Functional Theory, the Systemic Functional Multimodal Discourse Analysis approach to the formal and functional classification of teachers’ use of gestures in the classroom is introduced. The approach builds on present research and theoretical conceptions on gestures and extends the classifications with examples from a selection of instances from an authentic classroom corpus. The aim is to develop the theoretical apparatus for the annotation and analysis of the teachers’ use of gestures in the classroom.

The purpose of this study is to determine how World Health Organization (WHO) packaged information related to Covid-19 issue, especially in the campaign of vaccination program all over the world. The interactional meaning represented in World Immunization Week 2021 Posters used by WHO in the campaign is examined and analyzed. The analysis focuses on the representation of visual and verbal mode through a Systemic Functional Multimodal Discourse Analysis (SFMDA) approach; this approach makes it possible to complete the analysis by using not only visual grammar in determining the visual mode but also Systemic Functional Grammar in determining the verbal mode. By visual grammar, interactional meaning is analyzed through the concept of interactive meaning involving attitude, social distance, contact, and modality in both interactive and represented participants while by Systemic Functional Grammar, the meaning is analyzed through the concept of interpersonal meaning involving the analysis of mood system. The results reveal that a compact combination between visual mode and verbal mode may increase the viewers’ readability and appropriate knowledge to the message of the posters, and finally, the goals which are expected by WHO as the poster maker in the campaign can be communicated and understood well

A challenge for researchers working with multimodal classroom discourse is to be able to describe and discuss the interaction and interplay across various semiotic resources. This article adopts the Systemic Functional Multimodal Discourse Analysis approach to examine the relationship between language and gesture used by the teacher, in interaction with the students, and the emergent meanings made multimodally. It discusses the mechanisms by which language and gesture combine to make meaning by extending concepts originally developed for language–image relations. From the analysis and interpretation of the teachers’ multimodal selections in the lesson, the emergent meaning of ‘structured informality’ is proposed. Structured informality offers a way to consider how a teacher can design an effective learning experience for their students using multimodal resources.

Thematic progression plays a vital role in organizing the information in a text and in enabling it to be understood and communicated effectively. Studies of multimodal business discourse have been confined to workplace contexts, and across the fields of management accounting, marketing, and finance. Based on Halliday’s (2014) analytical tools of systemic functional linguistic (SFL) and Kress and van Leeuwen’s (2006) analysis of images, an Systemic Functional Multimodal Discourse Analysis (SF-MDA) was conducted to explore THEME and INFORMATION structure in five international students’ texts in a key topic in accounting, namely financial statements. The participants are first-year Master of Commerce Accounting Saudi students enrolled in the Accounting Concepts and Methods module at an Australian university. The findings of the SF-MDA revealed the frequency of Theme reiteration and the linear Theme pattern in financial statements. The first pattern is employed in accounting tables to list the corresponding numerical values. The SF-MDA findings of the balance sheet corresponded with Kress and van Leeuwen’s (2006) approach to the analysis of grammar of visual design in terms of compositional zones.

This study explores the representation of socio-cognitive components in Indonesian political multimodal news text using the framework of Systemic Functional Multimodal Discourse Analysis (SF-MDA). With the increasing prominence of multimedia in contemporary news production, understanding how socio-cognitive aspects are conveyed through multiple modes becomes essential. Drawing on SF-MDA, which combines systemic functional linguistics and multimodal analysis, the study examines how Indonesian political news texts incorporate social and cognitive elements to shape meaning and influence audience perceptions. This study was undertaken qualitatively involving three online news published by Indonesia’s publishers as the data. The result reveals that the visual and verbal modes merged in the news tend to highlight the good image of the political figures. It means that the meanings of visual and verbal components of the news persuasively work together to create the good impressions among readers to the human actor in the news. As the implication, multimodal resources are strategically employed to shape public opinion and promote specific political ideologies. Ultimately, this research aims to contribute to the development of critical media literacy and foster a more nuanced understanding of the socio-political dynamics in contemporary Indonesia.

Multimodality is the study of how multiple communication modes (e.g., language, gesture, and proxemics; language, image, and layout) combine to make meaning. Multimodality transcends and draws on a variety of disciplines and theories. One approach to theorizing multimodality builds upon the principles of systemic functional (SF) theory. One key aspect of SF theory, which has been widely applied in studies of multimodality, is the observation that there are three broad kinds of meaning (called metafunctions) in language. The three metafunctions of systemic functional multimodal discourse analysis (SF‐MDA) appear as representational, interactional, and compositional meanings in multimodal discourse. Researchers have used SF‐MDA to examine meaning in two dimensions (on page and screen), three dimensions (in physical space), and across time (in music and film).

This study explores the ideational meanings of multimodal texts in an EFL textbook entitled “Bahasa Inggris: When English Rings a Bell”. This study aims to describe the meanings of visual and verbal modes from ideational aspects. The visual modes were analyzed by using Visual Grammar Design framework (Kress van Leeuwen, 2006). Meanwhile, the verbal modes were analyzed using Systemic Functional Grammar theory focusing on the Transitivity system (Halliday, 1994). The ideational aspects of this study focus on components like participants, processes and circumstances. This study applied Systemic Functional Multimodal Discourse Analysis (henceforth SF-MDA) methodology. The findings of this study found that the dominant process types used in realizing the ideational meaning of the verbal text is relational process. Meanwhile, the ideational meaning of visual text is realized mostly through speech process. This study showed that the presentation of multimodal texts in the EFL textbook is relevant to the teaching and learning activity for the Junior high school students. Keywords: multimodality, ideational meaning, systemic functional grammar, multimodal analysis

Multimodal construction of ‘rule of law’ in Chinese anti-corruption public service advertisements

Culture and society are produced through interactions between people, objects, and environments. Within these interactions it has become clear that the modes of spoken and written language are only some of a diverse range of modes involved in producing meaning and experience. As topics of study, the modes of spoken and written language have been joined by modes like gesture, gaze, composition, and layout. Multimodal discourse analysis names a range of approaches to studying social interaction and meaning as multimodal, that is, produced with and through multiple modes. However, multimodal discourse analysis is not about identifying and studying modes as isolated but rather about understanding the world as multimodal. This understanding is developed through theoretical and methodological developments. Three major theoretical bases are in use in multimodal discourse analysis: (a) Hallidayan systemic functional linguistics (SFL), underpinning a social semiotic and SFL approach; (b) mediated discourse analysis, underpinning a mediated action based approach; and (c) conversation analysis (CA), underpinning a turn-taking based approach. The simple idea of identifying modes beyond language belies the growing complexity of research in this area. The multimodal literature contains significantly different definitions of mode and different foci. For example, systemic functional approaches began by analyzing how meaning is embedded within images and artifacts, and these studies have been extended at times to include studies of interaction. Mediated discourse analysis focuses primarily on interaction and understands images and artifacts through how social actors interact with them, rather than seeking to decode possible meanings from the perspective of the analyst. Conversation analytic approaches also focus on interactions and only engage with images and artifacts as they appear in interaction. All three theoretical bases develop from studies of language. Researchers in CA still often argue that talk remains the most important mode, while social semiotics and mediated discourse analysis dispute this. Multimodal (inter)action analysis builds upon mediated discourse analysis but is the only approach designed specifically for the study of multimodal interaction and multimodal action. The ongoing development of frameworks within multimodal discourse analysis is indicative of intellectual diversity. For some scholars, this is a source of concern, and they attempt to unify multimodality, while for others it is desirable, especially for a research domain that offers applications to so many areas of social life. This bibliography explores central texts under the umbrella of multimodal discourse analysis in five domains: mediated discourse analysis, multimodal (inter)action analysis, social semiotics, systemic functional multimodal discourse analysis (SF-MDA), and conversation analytic style multimodality. A sixth area focusing on introductory texts begins the bibliography.

Willingness to communicate (WTC) in a second language class has been shown to be transient and situation-dependent. Previous studies mostly elicited data from learners' self-reports through interviews or diaries, and the data analyses were largely confined to the medium of language, marginalizing other semiotic resources such as gesture or movement. The nature of human communication, however, is multimodal, which entails the processing of different semiotic resources. Drawing on a systemic functional multimodal discourse analysis approach, this study investigates the dynamics of WTC in the English as a foreign language classroom in China. The researchers collected data through videotaping English lessons, stimulated recall interviews, and learning journals. Their analyses involved two teaching scenarios videotaped in one class period, which were rated respectively with high WTC and low WTC by four participating students in stimulated recall interviews. These scenarios were transcribed and annotated using Multimodal Analysis Video software to identify the use of semiotic resources, including language, gesture, and gaze. The discourse semantic features of the two scenarios were analyzed and compared, and subtle differences were presented and discussed. This study's findings highlight the need for language teachers to recognize and orchestrate multimodal semiotic resources to enhance students' WTC and classroom participation.

ABSTRACTMultimodality has been a growing research interest in the field of applied linguistics. With the actor-network theory as an analytic framework, the purpose of the present study was to explore how university students of Teaching English as a Foreign Language (TEFL) take advantage of both material and non-material semiotic resources while being involved in the process of developing classroom activities. Furthermore, the semiotic resources of these students including their language, gaze, and gesture were analyzed using systemic functional multimodal discourse analysis (SF-MDA). To do this, using an ethnomethodology, we videotaped the interaction of a group of 6 TEFL students developing materials, for the improvement of learners’ intercultural competence, on the topic of food. The findings indicated that the participants developed these materials as they interacted with each other, as human semiotic resources, as well as their digital tools, as non-human resources. In addition, while mainly expressing their mental processes, they also conveyed positive attitudes and built negotiation expanding space using their gesture with both fast and slow graduation. Their gaze was also engaged, directed at both the group members and objects developing the classroom activities.

In our civic engagement afterschool program in the southeast of the United States, our youth and adult participants used a wide range of modalities including mapping, rapping, drawing, and performance to construct and convey their unique visions of school and community spaces. The purpose of this methodological paper is to explore systemic functional multimodal discourse analysis (SF MDA) as a potential resource to gauge the effectiveness of such humanizing and multimodal spaces in positioning youth as civic leaders. Data collected for this study included curriculum materials, descriptive field notes, and video recordings of the multimodal processes of focal youth across five program modules. Informed by an SF MDA perspective, our multimodal transcriptions illustrate how youth participants engaged in an intersemiotic complementarity of modalities and languages to share their insights and enact civic identities.

This research aims to investigate the use of social media in constructing political narrative by examining the video blogs (vlogs) of Indonesian President, Jokowi, on Youtube as a case study. How does President Jokowi use his vlogs to construct political narratives about his personal life and presidency? What kind of semiotic resources deployed in his vlogs? This research uses the method of Systemic Functional Multimodal Discourse Analysis and concludes that President Jokowi has used his vlogs to build his political narrative with the underlying themes of progress and reformation, stating that his presidency will lead Indonesia to a better situation. The political narrative championed in his vlogs strengthens his political legitimacy as a leader who is humble, modest, and close to the people. In building his political narrative, President Jokowi has used a variety of semiotic resources, including participants, process or kinesics action, locative circumstance, visual collocation, speech, and text. This research confirms the change of political communication in content dimension and shows the importance of applying multimodal analysis in understanding contemporary political communication. This research recommends politicians to use vlogs as a platform to build their political narratives in this new political environment.

Thematic progression and the composition of information value contribute to the development of well-structured meaningful text. Discourse-based research of the textual features has been confined to language learning and workplace contexts, and in particular advertising and marketing research. However, no published study has explored and analysed the multimodal textual and the logic-semantic features in tertiary marketing texts. This case study employed a Systemic Functional Multimodal Discourse Analysis (SF-MDA) to investigate and compare thematic progression patterns and composition of information value in five business marketing plan reports. It also aimed to investigate the logico–semantic expansions between the visual semiotic resources and the text surrounding them. The SF-MDA of the marketing plan texts is underpinned by Halliday’s (1994) systemic functional linguistics, Kress and van Leeuwen’s (1996) composition of information value and Martinec and Salway’s (2005) system for multimodal discourse analysis of logico–semantic relations. The findings of the SF-MDA revealed the extensive use of Theme reiteration pattern, followed by linear thematic progression pattern. Although multiple-Theme pattern was minimally employed, disciplinary-specific uses of this pattern in marketing discourse emerged. The findings suggest that the students have managed professional workplace practices by showing their understandings of marketing as part of overall business management. A number of other interesting findings related to textual and logico–semantic relations that exist between the tables and the graphs and the accompanying text were revealed. The SF-MDA of informational choices in the marketing texts extends Kress and van Leeuwen’s (1996) functional interpretations of visual artefacts in terms of compositional zones. As the intersemiotic logic-semantic relations between graphs and tables and the orthographic texts surrounding them play a vital role in marketing discourse, tutors can introduce ways of expanding the meaning-making potential in these semiotic resources. Other pedagogical implications for the teaching and learning of writing are presented, particularly in the teaching of English for Business students.