Social Semiotic Approach Research Articles

Exploring modality in multimodal macrogenre: A social semiotic analysis of EFL pedagogic materials in China

Abstract Given the evolving multimodal features in educational settings, modes other than language further enable the diversity in the realization of meanings and pedagogic goals. This paper explores modality in multimodal pedagogic materials for teaching English as a foreign language in China. Drawing upon the social semiotic approach to modality in visual media, this study provides a comparative analysis of modality markers in different elemental genres that constitute the macrogenre of a teaching unit, with a focus on explaining the underlying reasons for the different choices in terms of coding orientation. It is shown that different degrees of deviation from the accepted coding orientation are employed in different constituent genres of the macrogenre of a given text.

Complementarity of communicative modes on meaning making in Tanzania’s digital telecom marketing: A social semiotic multimodal perspective

This article examines the complementarity of the communicative modes for making meaning in digital telecom advertisements. Using a social semiotic multimodal approach in its data analysis, we utilise the metafunction and composition frameworks for the interpretation and discussion. The study shows that advert makers incorporate various modes which represent real-life objects and experiences to promote products and services. The modes establish and reinforce marketing and social relationships between makers and potential customers. In the adverts, visual modes are often elaborated on through texts. In composition, the study shows that advertisers utilise various patterns in organising the modes. The makers also use various compositional properties to indicate the prominence of the modes. In addition, they deploy discrete frames to separate units of information. As such, the combination of all these modes as an integrated system in the digital telecom advertisements enables the makers to negotiate meaning with consumers.

The Politics of Representation in Green Brand Communication: A Social Semiotics Approach

The Politics of Representation in Green Brand Communication: A Social Semiotics Approach

Gender, Islam and nativism in populist radical-right posters: visualizing ‘insiders’ and ‘outsiders’

ABSTRACT Sayan-Cengiz and Tekin explore the visual communication strategies of Western European populist radical-right (PRR) parties in disseminating nativist, anti-migrant and Islamophobic agendas through gendered visual representations. It is widely argued that the PRR homogenizes and dichotomizes both ‘native’ and Muslim migrant cultures through an ostensibly liberal discourse of respect for women’s rights and freedoms in order to mainstream their exclusionary position towards Muslim migrant communities. However, there is a void in the literature in terms of accounting for how visual communication strategies are used in this process. The authors here argue that looking into the visual representations of Muslim migrant and ‘native’ bodies in the PRR parties’ visual communication materials is crucial for understanding the PRR’s gendered cultural constructions of both Muslim migrant ‘outsiders’ and native ‘insiders’. Focusing on the campaign posters for the Alternative für Deutschland (AfD) in Germany, the Rassemblement National (RN) in France, and the Partij voor de Vrijheid (PVV) in the Netherlands using a social semiotic approach, their study suggests that Muslim migrants are represented as alien Others, but that there is also a significant difference between representations of migrant women as ‘victims’ and men as ‘aggressors’. On the other hand, ‘native’ women are represented as the embodiment of authentic national identities, as either reproducers or defenders of the nation. ‘Native’, heterosexual men do not often appear in the posters, pointing to the position of power that they hold in the PRR’s imagination. That is, they are the designated spectators and addressees of visual communication materials, rather than objects of representation.

Teaching for Emergent Disciplinary Drawing in Science? Comparing Teachers’ and Children’s Ways of Representing Science Content in Early Childhood Classrooms

AbstractThis classroom-based study aims to contribute knowledge about children’s opportunities to make use of drawing to make meaning in science. Employing a social semiotic approach to drawing, we examine what ways of representing science content that are (1) made available by the teacher and (2) adopted in children’s drawings. We analysed observation data from 11 science lessons in early childhood classrooms (children aged 3 to 8 years), including the drawings that children made during those lessons (129 drawings in total). Our findings suggest that the semiotic resources that teachers provide have a large impact on how children represent science content in their drawings. Moreover, we interpret that teachers strive to support children’s ‘emergent disciplinary drawing’ in science, since they predominantly provided semiotic resources where the science content was generalised and decontextualised. Finally, we propose that ‘emergent disciplinary drawing’ is incorporated as an element of science pedagogy in ECE practice and ECE teacher education.

Multimodal positioning of artifacts in interaction in a collaborative elementary engineering club

Abstract This exploratory case study uses multimodal positioning analysis to determine and describe how a purposefully crafted emergent artifact comes to influence and/or manipulate social dynamics, structure, and positionings of one design team comprised of five third-graders in an afterschool elementary engineering and literacy club. In addition to social semiotic theories of multimodality (e.g., Kress, G. (2010). Multimodality: a social semiotic approach to contemporary communication. New York, NY: Routledge) and multimodal interactional analysis (Norris, S. (2004). Analyzing multimodal interaction: a methodological framework. New York, NY: Routledge, Norris, S. (2019). Systematically working with multimodal data: research methods in multimodal discourse analysis. Hoboken, NJ: Wiley-Blackwell), Positioning Theory (Harré, R. and Van Langenhove, L. (1991). Varieties of positioning. J. Theor. Soc. Behav. 21: 393–407) is used to examine group interactions with the artifact, with observational data collected from audio, video, researcher field notes, analytic memos, photographs, student artifacts (e.g., drawn designs, built designs), and transcriptions of audio and video data. Analysis of interactions of the artifact as it unfolds demonstrates multiple types of role-based positioning with students (e.g., builder, helper, idea-sharer). Foregrounding analysis of the artifact, rather than the student participants, exposed students’ alignment or opposition with their groupmates during the project. This study contributes to multimodal and artifactual scholarship through a close examination of positions emergent across time through multimodal communicative actions and illustrates how perspectives on multimodality may be analytically combined with Positioning Theory.

Assessment, Recognition and the “Contact Zone” in Landscape Architecture: How Much Is “Enough”?

Extended Curriculum Programmes have a responsibility to validate the resources and experiences students bring to their learning environment. However, designing assessment practices that encourage diverse students to draw on their resources in order to both access and challenge disciplinary discourses can be complex. This article is framed in terms of how students balance their own experiential knowledge while engaging with the disciplinary discourse. It aims to interrogate students’ negotiation of the “contact zone” and how they negotiate their brought-along resources with assessment guidelines. A multimodal social semiotic approach is taken to explore ways of contributing to a socially just pedagogy by enabling recognition of a range of students’ resources, while at the same time acknowledging the need to access the conventions of the discipline. We argue for recognition as the positive side of assessment, which could enable more diverse students’ resources to be acknowledged. We interrogate the meaning-making trajectories of two students, Xola and Sonwabo, in a first-year landscape architecture course. While both students bring their own resources into a spatial model project, they each have varying “success” in mediating these in relation to the dominant conventions of landscape architectural design.

Exploring digital interactivity: a multimodal and social semiotic approach to intersemiotic construction in COVID China website

Abstract This paper explores intersemiotic construction and digital interactivity through a multimodal social semiotic lens examining the semiotic instantiations of interactive signs on the homepage of “Fighting COVID-19 the Chinese Way” (covid-19.chinadaily.com.cn), a website created to disseminate information on news and development of COVID prevention and control practices in China to contribute to global efforts to fight the pandemic. The study focuses on how digital interactivity is afforded by the COVID China website, where the interactive signs are ideationally and compositionally constructed for meaning representation and interpersonally for text-viewer relation construal. From the multimodal social semiotic perspective, systemic functional model-based cluster and intersemiotic analyses are applied to explore the visual, spatial, and linguistic features that contribute to the design and construction of interactive semiotic signs on the COVID China website and afford digital interactivity for viewers’ action potentials. This study extends the analytical focus to the semiotic instantiations of interactive signs and their intersemiotic construction process that stimulate the enabling of interactivity, instead of the interactivity per se, and demonstrates how different semiotic instantiations of interactive signs are featured and interact to afford digital interactivity. It argues for an integrated lens in analysis to look at the interactive signs not only as signs of action with action-enabling forms but also as signs of meanings that afford user-page interactivity.

Corporate branding of academic institutions: semiotic communication of logos and names

ABSTRACT This study focuses on the role of visual resources in constructing the brand identity of Chinese universities. By drawing on the theoretical concept of corporate visual identity (CVI), this study analyzed universities’ logos and typography of names from the social semiotic approach. Nine elite Chinese universities were selected as the corpus to qualitatively investigate how the institutions’ logos and typography of their names are designed and utilized for corporate branding. The analysis shows that both the logos and names are graphically and typographically designed as effective visual elements and semiotic resources with meaning potentials to signify the Chinese universities’ brand identity.

The Interplay of Text and Image on the Meaning-making Processes of Adult L2 Learners with Emerging Literacy: Implications for Test Design and Evaluation Frameworks

ABSTRACT This study investigates meaning-making processes in language and literacy assessments. Using a social semiotic perspective, it examines how adult second language learners with emerging literacy self-articulate their understanding of multimodal elements and components utilized in low-stakes assessments, and the strategies they use to make meaning in such tests. The study examines how the intended meaning of assessment prompts aligns with the perceived meaning on the part of participants. To do so, we used an existing assessment tool developed for refugee-background adults in the United States as well as two experimental assessment texts created as part of the research. Data was collected via semi-structured interviews with 14 participants. Interviews were analyzed using a critical multimodal social semiotic approach, informed by systemic functional linguistics. Data was coded according to metafunction and theme, with results organized by test genre element and multimodal component. Results exposed tensions between participants’ responses to textual and visual prompts and the expectations of test designers. Findings reveal that textual composition and assessment practices may be inadvertently biased against individuals with diverging literacy profiles. Results have implications for assessment of this population such that language, literacy, and test socialization should be considered during test design and evaluation framework development.

Making sense of the “raw meat”: A social semiotic interpretation of user translation on the danmu interface

The danmu interface is a unique feature on Japanese and Chinese video-sharing websites which allows “live” comments to be directly overlaid onto the video. This paper focuses on one particular semiotic and social practice materialised on the danmu interface, collaborative user translation, where users use their danmu entries to post translations onto the video frame in their engagement with the “raw meat”, untranslated foreign language videos. Adopting Theo van Leeuwen’s social semiotic approach to visual communication and his conceptualisation of “semiotic regimes”, this study explores how such a translation practice materialises on the danmu interface through an investigation of the use of semiotic resources in relation to relevant semiotic regimes that derive from the semiotic technology of danmu, professional practices of audiovisual translation and relevant regulatory discourses. Multimodal discourse analysis is conducted on a dataset consisting of an untranslated English documentary video posted on Bilibli, the most popular danmu video website in China, and 1973 entries of danmu comments archived on the interface. This paper contributes to the development of social semiotics-oriented multimodal discourse analysis through a focus on the semiotic and social practice of user translation on danmu, adding more depth to existing studies on such a practice, and broadens the scope of non-professional translation studies.

WHITE SUPREMACISTS DECEPTIVELY USING SCREENSHOTS AS EVIDENCE: A SOCIAL SEMIOTIC APPROACH TO ANALYSING CONSPIRATORIAL YOUTUBE VIDEOS

YouTube has become notoriously associated with extreme right-wing communities that spread discourses of white supremacy and conspiracy. This study applies a social semiotic approach to analysing conspiratorial YouTube videos created by white supremacists in response to the Notre Dame Fire of 2019. In particular, this study applies a combined legitimation (Van Leeuwen, 2007) and communing affiliation (Zappavigna and Martin, 2018) framework to the verbal and visual content of 15 videos. Communing affiliation refers to how values are positioned as bondable in situation where users don’t interact directly (Zappavigna and Martin, 2018). It is formed from couplings of ideational (what is being evaluated) and attitudinal (how it is evaluated) meaning (Martin and White, 2005), hence forming a value that is bondable. Legitimation (Van Leeuwen, 2007) refers to how discourses establish authority and can be realised textually or visually, with various linguistic strategies. This study will focus on the idea of ‘technological authority’ construed by positive evaluations in the transcripts of screenshots as evidence and the use of screenshots as visual evidence. Overall, this study will show how key values are working in tandem with (de)legitimation strategies, how (de)legitimation can further explain the significance of these values, and how YouTubers artificially create credibility in their videos through these legitimation and affiliation strategies. This raises further questions about the invoked meanings of screenshots as evidence and the ethical dilemmas that screenshots become entangled in, when considering the attention given to false and hateful content that is shared online.

Learning Ecologies in the Digital Era: A Challenge for Primary Education in Low-Income Contexts

This research focused on the study of learning ecologies utilizing digital technology. The qualitative methodology used has allowed the analysis of the interactions children establish with digital technologies and the manner in which they construct a learning ecology. A total of 46 12-year-olds, their families, and their teachers participated in this study. The children’s schools and homes were in neighborhoods where structural situations of poverty and social and cultural marginality concurred. The children integrated researchers into their digital community, which allowed access to the events that the community was developing through digital technologies for two years. This information was complemented by the development of systematic observations and interviews with each participant. The participants’ multimodal linguistic and literacy practices were analyzed using a social semiotics approach. The results of the research describe and interpret the interactions that took place between participants and digital technologies. The research has identified the processes of recontextualization, transduction, and transcontextualization of the discourses developed in the frame of the participants’ learning ecologies. Digital ethnography has been revealed as an adequate method for studying learning ecologies.

Cultural Representation in EFL Textbooks for the Seventh Graders: A Multimodal Analysis

This research aimed to investigate the types of cultures represented in the EFL textbooks for the seventh graders. Two EFL textbooks were analyzed: ‘When English Rings a Bell’ and ‘English on Sky 1’. This research used qualitative content analysis as method of research and applied Kress and van Leeuwen’s multimodal social semiotic approach as the technique of analyzing the data. The result reveals that imbalances in the representation of cultural types exist. Source culture is represented more dominantly compared to target and international culture. It implies that both the authors and the publishers of the textbooks prioritize the learners’ identities construction and characters education rather than focus on the use of language as the lingua franca. Consequently, it could constrain the opportunities for learners to acquire international cultural knowledge and become successful intercultural communicators. Therefore, English teachers need to take an active role in selecting appropriate textbooks and adding supplementary materials or activities to patch up the shortcomings of the textbooks.

A Semiotic Approach to Understanding Advertisements

Abstract This research investigates the role of semiotics in comprehending the ideological and cultural symbols utilised in advertisements. The major aim of this study is to examine how the different approaches of semiotics contribute to interpreting the messages decoded in advertisements. Under the broad theory of semiotics, the current research presents Saussure's (1916) dyadic model of the sign as well as Peirce's (1955) triadic model of the sign. Furthermore, this research highlights the predominant role played by advertisements in our society these days. The study also gives an account of Barthes' (1977) categorization of the messages existing in advertisements into linguistic, denotative and connotative. In addition, an overview of van Leeuwen's (2005) social semiotic approach is presented. Findings of the study indicate that advertisements are loaded with signs whose interpretations depend largely on culture and ideology. The study also shows that the aforementioned semiotic approaches can help in revealing how advertisements perpetuate certain ideas and beliefs while ignore others. It indicates that most advertisements try to communicate implicitly the idea of how people are supposed to be. مستخلص البحث باللغة العربية تبحث هذه الدراسة دور السيميوطيقا في فهم الرموز الأيديولوجية والثقافية المستخدمة في الإعلانات. والهدف الرئيسي لهذه الدراسة هو فحص کيفية مشارکة المناهج المختلفة للسيميوطيقافي تفسير الرسائل المکنونة في الإعلانات. ففي ظل نظرية السيميوطيقا تقدم هذه الدراسة نموذج سوسير (1916) الثنائي للعلامة بالإضافة إلى نموذج بيرس (1955) الثلاثي للعلامة. وعلاوة على ذلک تسلط هذه الدراسة الضوء على الدور السائد الذي تلعبه الإعلانات في مجتمعنا هذه الأيام. وتقدم الدراسة أيضا سردا لتصنيف بارثيس (1977) للرسائل الموجودة في الإعلانات إلى رسائل لغوية ، رسائل دلالية، رسائل ضمنية. فضلا عن ذلک يتم عرض لمحة عن نهج فان ليوين (2005) عن السيميوطيقا الاجتماعية. وتشير نتائج هذه الدراسة إلى کون الإعلانات حافلة بالعلامات التي تعتمد تفسيراتها بشکل کبير على الثقافة والأيديولوجيا ، کما توضح هذه الدراسة أن المداخل السيميوطيقية سالفة الذکر من الممکن أن تساعد في کشف کيفية ترسيخ الإعلانات لمفاهيم وأفکار بعينها في حين تجاهل الأخرى. وتوضح أيضا أن معظم الإعلانات تحاول ضمنيا توصيل فکرة عن الشکل المفترض الذي ينبغي أن يکون عليه الأشخاص.

Ideological Representations in the Nomination Acceptance Speeches of Donald Trump and Hillary Clinton: A Contrastive Study

The study examines the correlation between language and ideology through critical discourse analysis and semiotics of two nomination acceptance speeches of the 2016 American election. It is an interdisciplinary study that combines different fields: ideology, ideological structures, and semiotics. Through a comparative analysis between the speeches of two nominees who run for the American presidency: the Republican candidate, Trump, and the Democratic candidate, Clinton, the study explores the ideological polarization of the self and Other presentations that each candidate applies concerning three themes: internal problems, foreign affairs, and immigration. Besides, the internal problems theme is sub-classified into five sub-issues: crime, violence, economy, American rights, and health care. In this regard, the study focuses mainly on uncovering the hidden ideologies that each speaker utilizes either verbally and nonverbally to reveal how far each speaker can successfully manipulate the language and utilize signs to serve his/her political intentions. Therefore, the ideological square model of Dijk (2006a) is utilized for the verbal analysis to provide an analysis of polarization techniques of the self and Other. The ideological structures and strategies model of Dijk (2006b) is also applied to do a linguistic analysis along with Kress (2010) multimodal social semiotic approach for the nonverbal analysis. The study revealed that Trump and Clinton are varied in presenting self and Other presentations. For the nonverbal analysis, Trump concentrated more on denigrating the negative Other-presentation than boasting the positive self-presentation. On the other hand, Clinton focused on lauding the positive self-presentation than disparaging the negative Other-presentation. However, they both agreed on neglecting the mention of the good properties of the out-group and the bad properties of the in-group. Concerning to the nonverbal analysis, Trump and Clinton also utilize more semiotic modes to boast the self-presentation and perform fewer semiotic modes that denigrate the Other-presentation. On the contrary, Clinton focuses on emphasizing the positive self-presentation than disparaging the negative Other-presentation. Besides, the study introduced a modified model that helps to analyze nomination acceptance speeches verbally and nonverbally.

The Communicative Function of Non-Face Emoji "Folded Hands" in WhatsApp Interaction among Indonesians

This study aimed to examine the communicative functions of the non-face “folded hands” emoji that Indonesians use in WhatsApp interactions. Using descriptive qualitative, the data were analysed through the social semiotic approach proposed by Lemke (1998), the interactional sociolinguistic theory (Gumperz, 1982), and speech act theory. The data were taken from the messages that occurred naturally in several WhatsApp groups. It was re-transcribed and translated in English based on the theory by Dresner and Herring (2010). The findings revealed some functions of the non-face “folded hands” emoji in online communication. At the end of an utterance, it emphasizes message tone, politeness, and soft interaction. Furthermore, rather than this emoji expresses an emotion such as face emoji does, it performs illocutionary forces in communication such as thanking, apologizing, and asking. It is also performed in situational interaction in which the relation of speaker and interlocutor has a formal relation such as teacher-students and leader-staffs. This empirical research added to computer-mediated discourse literature by analyzing the communicative function of folded hands emoji in written discourse. Practically, it might help users interpret the emoji as an abstract concept in the messages.

Visual And Metaphorical Representation In McDonald’s Singapore Advertisement

<p>The raise of technology advanced the advertising platform, social media, as one of their ways of offering products that can reach wide audience around the world. This research aims to scrutinize the advertisement of McDonald’s Singapore Instagram account using social semiotic approach and pictorial metaphor theory. This research applied four steps of collecting and analysing 60 multimodal data 1) collecting the data in McDonald’s Singapore account, 2) observing the data, 3) taking data sample, and 4) transcribing and analysing the data. From the analysis, the result of this research shows that conceptual representation dominates the data by 39 data and it is followed by embedding and narrative representation by 19 and 2 data respectively. The compositional meaning was combined with the analysis of pictorial metaphor and the result shows that the non-metaphorical centre is more dominant that others by 15 data, and then it is followed by non-metaphorical tryphtic as many as 13 data, non-pictorial metaphor ideal-real by 8 data, pictorial metaphor tryphtic by 6 data and pictorial metaphor centre by 5 data, while the rest is pictorial metaphor given-new, non-pictorial metaphor given-new, pictorial metaphor ideal-real, and verbo-pictorial metaphor ideal-real by 5, 4, 2, and 2 data respectively.</p>

Qualitatively different ways of unpacking visual representations when teaching intermolecular forces in upper secondary school

AbstractSince visual representations play a particularly important role in the teaching and learning of chemistry, the exploration described in this article focuses on them. This is an explorative study of the qualitatively different ways that visual representations can be unpacked by Swedish upper secondary school chemistry teachers dealing with intermolecular forces. Unpacking is characterized as the ways that visual representations get used to open up the possibility of having the critical aspects and features of an intended object of learning being brought into focal awareness, initially on their own and then simultaneously. The analysis, which combines a phenomenographic and a social semiotic approach, leads to the characterizations of five qualitatively different ways that visual representations may be unpacked. These outcome categories are presented in terms of a conceptual hierarchy, where two of these ways of unpacking are characterized as being teacher‐centered and the other three as student‐centered. This leads to a case being made that if teachers use student‐centered ways of unpacking visual representations, then their students will be more likely to gain greater access to critical aspects and features of the enacted object of learning. We argue that in terms of making theoretical and practical contributions to the phenomenographic perspective on learning, the results can be used as a tool for researchers wishing to explore how visual representations can be used effectively in science education and also provide a useful basis for discussion in teacher education and in teacher professional development programs.

Women and fitness on Weibo: the neoliberalism solution to the obligations of Confucianism

ABSTRACT This paper uses a social semiotic approach to analyse the posts of women fitness experts/influencers on Chinese Weibo. On the posts, we find a harmonized world where all parts of life can be managed by making the right choices and by having a striving attitude. Here, success and happiness become tasks to be worked on. Yet this is a decontextualized world, where there is no room for actual situations and dispositions. We discuss how these representations can be related to rising neoliberal ideas, values, and identities among the new Chinese middle classes, which, in these instances, are used to create a rather overdetermined stance against more traditional Confucian women’s roles with an emphasis on caring, knowing one’s position, and kinship obligations. But the one-size-fits-all rhetoric of empowerment, getting-ahead and choice leaves little room for sharing the actual restrictions, conflicts, and struggles faced by these women.