Integrating audio–visual text generation with contrastive learning for enhanced multimodal emotion analysis

Analyzing events in dynamic environments poses a fundamental challenge in the development of intelligent agents and robots capable of interacting with humans. Current approaches predominantly rely on visual–text models; however, these methods often capture information implicitly from images, lacking interpretable and structured spatio-temporal object representations and their relationships. To address this issue, we introduce DyGEnc—a novel method for dynamic graph encoding. This method integrates compressed spatio-temporal representation with the cognitive capabilities of large language models. The purpose of this integration is to enable advanced question answering based on sequences of textual scene graphs. Extensive evaluations on the STAR and AGQA datasets demonstrate that DyGEnc improves large language model performance when addressing queries related to the history of human–object interactions. Furthermore, the proposed method can be extended to process input images by leveraging foundation models to extract explicit textual scene graphs, as validated by the evaluation results. We expect these findings to contribute to the development of robust and compact graph-based memory for long-horizon reasoning in real-world applications, as demonstrated in a robotic experiment conducted using a wheeled manipulator platform.

Non-invasive Magnetic Resonance Imaging on clinical MR-scanners at spatial resolution ≂1mm3 could in principle be used also for insights in the structure of valuable ancient objects like books if the spatial resolution could be improved. We demonstrate, that Magnetic-Resonance-Microscopy is even able to visualize printed letters at thickness < 30 µm on superposed paper-sheets. The physical-technical methodology is based on prototype hardware installed as insert on a human MR-scanner relying on strong magnetic and gradient-fields with sensitive radiofrequency-sensors. A negative contrast mechanism, adding MR-visible chemically-inert liquid, is necessary, its removal being potentially harmful to valuable paper sheets. For visualization of text firstly the letter thickness (≂20 µm) and exact positions were determined before adjusting optimized resolution and field-of-view (FOV) in measurement and reprocessing of 3D-data for slice-positioning in the plane of the paper sheets. The advantages of a semi-automatic data processing method for visualization on the bowed paper-sheets are demonstrated. In contrast to micro-Computed tomography based on absorption contrast, MR-microscopy can visualize non-metallic printed letters and offers higher spatial resolution than Terahertz imaging. As a preclinical MR-imaging tool (with limited FOV) it is more widely available than neutron-tomography. The reported MRM-technology might be also of interest for radiological high-resolution applications such as MR-based histology.

ABSTRACT This paper examines Benh Zeitlin’s Beasts of the Southern Wild and Shaunak Sen’s All That Breathes as visual texts that engage the ecological and affective contours of Anthropocene cinema. Drawing on Rob Nixon’s concept of ‘slow violence’ and Lauren Berlant’s notion of ‘lateral agency,’ it considers how both films render visible the lived experiences of marginalised human and non-human subjects navigating environmental collapse under capitalist modernity. Through close visual and formal analysis, the paper argues that the films construct a poetics of survival rooted in repetitive labour, sensory attunement and mythic imagination. The characters in both narratives are depicted as practising self-abeyance and slow acts of repair – maintaining decaying ecologies rather than seeking escape from intolerable, precarious conditions. In doing so, the films make visible the often-invisible operations of structural violence, which manifest not in spectacle but in the lingering attrition of slow death, while simultaneously carving out fleeting, embodied moments of agency and ecological kinship.

This study focuses on the important visual text "A Mirror for the Emperor", exploring how it transforms Confucian "rites" into social norms through visual mechanisms. Unlike previous studies that mainly remained within the fields of textual analysis or political history, this paper approaches it from the perspective of visual culture and etiquette dissemination, analyzing its unique value as a "visual etiquette mechanism". Methodologically, this study combines Panofsky's three-level analysis method of iconography, visual semiotics, and political etiquette theory to conduct in-depth interpretations of the three core images: "Appointing Talents for Governance", "Warning Stelae and Criticism Boards", and "Filial Piety Ascending to Heaven". At the same time, this study pays attention to its regional reconfiguration in Shandong, Henan, and Shaanxi, revealing the recontextualization process of these images in local academies, temples, and families. Additionally, this study draws on Cosmological Ethics and normative social psychology to examine how these visual symbols construct the paradigm of normative compliance. The research on Ming Dynasty etiquette images also confirms that the visual presentation of natural symbols reinforces society's understanding of "the harmony between heaven and man", thereby extending the political admonition consciousness to a widespread reverence for cosmic order and social order. The research results indicate that the visual mechanism of "A Mirror for the Emperor" mainly manifests in four aspects: Firstly, through elements such as postures, objects, and spaces, it constructs a stable visual grammar, making "rites" have systematic expressibility; Secondly, the images align with the political tensions of the Wanli period and become symbols of admonition; Thirdly, the regional re-production realizes the cycle of "central generation - local adaptation", promoting the diffusion of the ritual system in grassroots society; Fourthly, the visual medium transforms ethical norms into daily learnable postures, strengthening social integration. The academic contributions of this study can be summarized as four points: expanding the artistic history perspective beyond textual studies; emphasizing the cultural function of etiquette in visual communication; integrating iconography and etiquette theory to reveal the interaction mechanism between visual and power; and placing Chinese etiquette images within the academic discourse of global visual culture and cross-cultural comparison.

ABSTRACT In an increasingly data-driven world, supporting adults in developing data literacy is essential for informed participation in society. For learning to be relevant and impactful, adults need to experience a sense of agency in the process. Game-based learning provides a safe, engaging, and experiential space that enables adults to explore and develop data-related knowledge and skills in meaningful, situated ways. Building on this premise, the Erasmus + project Data Literacy for Citizenship (DALI) explored how game-based learning experiences, originally designed to foster Data Literacy, promote agency among adult learners in both formal and informal continuing education contexts. A cross-national sample of 338 participants from Spain, the UK, Germany, and Norway played purpose-designed DALI games and completed an online survey. Using a quantitative, descriptive–correlational design, we analyse how individual factors (age, prior gaming experience) and perceived game quality shaped opportunities for enacting agency within these learning environments. Results show that the game-based strategies of the non-digital DALI games allowed agency activation, regardless of adult age and game experience. The perceived fun, along with game quality, and the clarity of visual texts emerged as drivers of agency in game-based learning. Implications for instructional design, game development and policy are discussed.

The reading comprehension ability of third-grade elementary school students is still low despite the implementation of various conventional teaching strategies. This gap hinders the achievement of the Pancasila Student Profile in the Merdeka Curriculum. This study analyzes the effect of the Cooperative Script learning model assisted by Canva animation media on the reading comprehension ability of lower-grade elementary school students. A quasi-experimental study with a Non-Equivalent Post-Test Only Control Group Design was used in this study. The research subjects consisted of 47 third-grade elementary school students, with 29 students in the experimental group and 18 students in the control group. Data were collected through a 20-item multiple-choice test measuring Bloom's taxonomy levels C1-C6 in narrative texts, validated for content by two experts, and tested for reliability using KR-20. Data analysis included descriptive statistics, Shapiro-Wilk normality test, Levene's homogeneity test, and independent t-test with pooled variance. The research results analyzed a significant increase in reading comprehension skills in the experimental group that received the Cooperative Script model assisted by Canva animation compared to the control group. The visual collaborative model proved to optimize comprehension from literal to critical levels through alternating oral summaries and dynamic text visualizations. This study concludes that integrating the Cooperative Script with digital animation media is effective in closing the elementary literacy gap, supporting essential learning in the Independent Curriculum. Its implications include producing ready-to-use teaching modules for Indonesian language teachers and recommendations for digital literacy policies in elementary school clusters.

Neural correlates of form- and meaning-based predictability during speech perception.

Optical Character Recognition (OCR) transforms visual text into machine-readable form, supporting the large-scale digitization of printed, handwritten, and scene-based documents. Early approaches, such as template matching and motion analysis, relied on handcrafted patterns and were constrained to limited fonts and simple layouts. The introduction of statistical models, including Hidden Markov Models and Conditional Random Fields, expanded OCR capabilities through probabilistic sequence modeling. With the rise of deep learning, Convolutional and Recurrent Neural Networks enabled end-to-end recognition, reducing dependence on manual feature engineering and improving performance on noisy or cursive text. More recently, transformer-based models like TrOCR have redefined OCR by leveraging self-attention and large-scale pretraining, achieving state-of-the-art results across multilingual and domain-specific applications. These models excel in cross-lingual transfer, low-resource adaptation, and specialized domains such as biomedical and historical text recognition, while integrating pretrained vision–language components for greater robustness against degraded inputs. Despite these advances, challenges persist in adversarial robustness, complex document layout understanding, and fairness across underrepresented languages and scripts. Emerging research directions include zero-shot and few-shot learning, modular adapters for scalable multilingual OCR, post-OCR correction pipelines, efficiency improvements, and privacy-preserving inference. This survey outlines OCR’s historical progression, highlights deep learning and transformer-based breakthroughs, and points to future work needed to address enduring challenges in this critical field of document analysis.

ABSTRACT Artificial Intelligence (AI) art generators are impacting the way screen content is produced, and this has implications for the way creative processes are taught in educational contexts. In this study, the use of AI to create screen content was investigated through the perspectives of students in a university course, which required them to transfer meaning across modes with and without AI art generators. This process of transmediation with and without AI was explored through the blogs and reflective reports submitted by students as part of their assessment tasks. The study showed that transmediating linguistic texts into visual texts was a meaning-making process that involved interpretation and creativity. The AI art generator could become a collaborator in these interpretive and creative processes, but did not match the capacity of humans to interpret deeper metaphorical meanings or create authentic and intentional texts. The use of AI had a varied impact on how students felt about the activities as they reflected on the images that they were creating. This study contributes to our understanding of the use of AI in the interpretation and creation of texts and has implications for the way students use AI in their practice.

This paper develops a theoretical and interpretive framework for analyzing a distinct type of urban surface – what I call the architectural inscriptional surface. These are architectural features that have undergone acts of removal, repair, sealing, or exposure, such that they visibly record traces of construction, decay, and other material events. Architectural inscriptional surfaces are placially anchored – they are not interchangeable surfaces, but deeply situated in the cultural, historical, and material specificity of their site. When photographed, they become visual texts and inscribed social objects: layered surfaces that invite cultural, historical, and ideological interpretation. Drawing on Roland Barthes’ theory of photographic meaning and Maurizio Ferraris’ ontology of documentality, the paper explores how inscriptional surfaces function simultaneously as aesthetic images and socially inscribed objects. Through a case study of a single photograph (The Gated Doorway) I demonstrate how Barthes’ three photographic messages and Ferraris’ concept of the social object reveal the multilayered nature of inscriptional surfaces as material signs, ideological symbols, and cultural documents. Situated within a lineage of architectural and urban photography that foregrounds entropy, rupture, and inscription, this paper offers a new model for understanding how surfaces remember – and how photography renders those memories legible.

This study aims to explore the potential and direction of literacy education in the field of art education, in which learners meet people, situations, phenomenology, and realities that learners confront, beyond cultivating the ability to use media in the field of art education. Five directions for developing literacy education in art subject education are proposed. The first direction is reading the intrinsic form of visual text, reading context, learners' self-reading and self-writing. The second is to include the self-care concept of taking care of oneself and careing for others. Third is to recognize literacy as social competence.The fourth and fifth direction is to include a multiliteracy concept that means the ability to build and understand reality and digital reality, and the physicality of the 'body'.

This article examines the concept of an immersive legal dictionary, its distinction from traditional legal dictionaries, and its role in legal education. The effectiveness of using immersive technologies (VR, AR, multimedia) in mastering legal terminology is analyzed from the perspectives of cognitive linguistics and digital pedagogy. Based on international experience, the study identifies the cognitive, pedagogical, and professional advantages of the immersive dictionary and substantiates its potential for developing professional proficiency in the Kazakh language. The immersive legal dictionary represents an innovative direction in legal lexicography. The research established that the static, referential nature of traditional legal dictionaries is insufficient for effective terminology acquisition. The digital technologies of the 21st century, including virtual reality (VR) and augmented reality (AR), provide opportunities to enhance the process of legal education. An experiment conducted at I. Zhansugurov Zhetysu University demonstrated the high effectiveness of immersive dictionaries in helping students master legal terminology, increase learning motivation, and develop professional skills. Immersive technologies enable information to be perceived through multiple channels, providing cognitive and pedagogical benefits. The structure of the experimental class was aimed at comprehensive contextual learning of terminology and included a terminology review, immersive tasks, reflection, and case studies. Moreover, the immersive dictionary contributes to the visualization and interactive presentation of complex legal texts and the formation of legal thinking. The global experience of integrating immersive technologies into legal education can serve as a model for Kazakhstan and promote the professional development of the Kazakh language. The research results confirm that immersive lexicography marks a transition to a new qualitative stage in legal education.

Caricatures are not merely visual humor but function as powerful instruments of socio-political critique. This study analyzes three caricatures by Osama Hajjaj that depict the condition of Syria following the collapse of Bashar al-Assad’s regime. Guided by Roman Jakobson’s semiotic framework, the analysis applies six language functions: addresser, addressee, message, context, code, and contact to explore the layered meanings embedded in Hajjaj’s works. Jakobson’s model is relevant for visual texts because its communicative functions operate through both verbal and non-verbal signs, allowing caricatures to be interpreted as structured messages. A qualitative approach was employed, with primary data obtained from caricatures published on Hajjaj’s Instagram account and secondary data drawn from scholarly literature and credible news reports. The results demonstrate that Hajjaj’s caricatures transcend mere satire of Assad’s leadership; they articulate broader grievances regarding foreign intervention, systemic repression, and the uncertainties surrounding political transition. Furthermore, the visual narratives reveal how caricatures can simultaneously criticize power structures and express collective resistance. This research contributes to the field of semiotics by expanding Jakobson’s framework into the analysis of political caricatures and provides new insights into visual communication in Middle Eastern contexts. The novelty of this study lies in its focus on Syrian-themed caricatures, a topic rarely examined in previous research, and in its analysis of the interaction among Jakobson’s functions to reveal the ideological dimensions within Hajjaj’s works.

The Ural code, represented by P. P. Bazhov’s tales, is examined in the article as a representative of regional culture at the level of verbal and visual texts. The illustrations for the editions from different years carry a special semantic load, demonstrating the intricacies of the dialogue with the Ural writer, time, and space, both real and fictional. Based on a hermeneutical approach, the article highlights the stages of reading, comprehending, and understanding a literary work, and places emphasis on the activities of illustrators.

Purpose This study aims to examine cultural and ecological representations in an Indonesian high school language textbook, exploring how cultural preservation and environmental sustainability are portrayed to support character education and civic values. Design/methodology/approach Guided by a critical paradigm, this research adopts a critical discourse analysis approach framed by cultural ecology and ecological systems theory. Narrative, descriptive, and visual texts in the textbook were analyzed using Fairclough’s critical discourse analysis model to identify patterns and thematic constructions of cultural and ecological discourse. Findings The analysis reveals the presence of several interconnected themes, including ethnic diversity, traditional arts, local technologies, and environmental stewardship. Illustrations of traditional dances, indigenous transport, and climate-action activities function as pedagogical tools that foster cultural identity, environmental awareness, and responsible citizenship. The findings highlight the textbook’s role in embedding local wisdom and ecological values within the national curriculum. Originality/value This study contributes to the understanding of how high school language textbooks construct cultural–ecological knowledge by integrating cultural preservation with sustainability discourse. It underscores the value of visual and textual representations as agents of sociocultural harmony and environmental consciousness, and offers implications for future research on classroom reception and long-term impacts on students’ ecological and cultural literacy.

INTRODUCTION . Intervisuality as a strategy for creating and transmitting meanings in modern advertising discourse has been investigated. The genesis of the concept, which emerged as a development of intertextuality theory in the context of the “visual turn”, is considered. Intervisuality is defined as a strategic creative technique based on the inclusion of recognizable visual elements in advertising that refer to art precedents. The aim of the study is a comprehensive analysis of intervisuality as a strategy for constructing meanings and emotional connections with the audience in the context of an oversaturated visual media space. MATERIALS AND METHODS . The research material consisted of a corpus of 105 visual texts from modern advertising, selected based on the presence of intervisual references. The methodological basis includes semiotic, intertextual, discourse analysis, comparative, and cultural-historical methods, which ensured a comprehensive study of the strategy based on intervisuality. RESULTS AND DISCUSSION . The main types of visual precedent phenomena used in advertising have been identified and systematized: precedent names, texts, situations, and artistic techniques (surrealism, pop art, cubism, etc.). The functions of intervisuality are demonstrated: attracting attention, meaning-making, forming brand identity, and targeted impact on the audience. Particular attention is paid to the risks associated with ethical violations, the desecration of cultural heritage, and the possibility of misinterpretation. Examples of advertising campaigns demonstrate both successful and controversial applications of intervisuality. CONCLUSIONS . Intervisuality has been found to be an effective tool for creating multidimensional messages, but its success depends on the appropriateness of choosing a visual pretext and the literacy of its integration into an advertising message.

This study aims to analyze the iconographic program of faunal and mythological figures found in the Candi Bumi Ayu temple complex, South Sumatra. As the only major Hindu-Shaivite site from the classical period in the region, active between the 9th and 13th centuries AD, its artistic representations offer a unique perspective on the religious and cultural dynamics within the Srivijayan sphere of influence. Through a descriptive-analytical method, this paper examines thirteen specific iconographic elements, including the Kala head, Makara, Nandi, Gana, and various faunal reliefs. The findings indicate that these figures are not merely decorative but constitute a coherent symbolic system with three primary functions: apotropaic (guarding sacred space), theological (representing the divine presence of Shiva and his retinue), and cultural (articulating a distinct local Sumatran artistic canon). This local style, while in dialogue with Javanese and Indic traditions, demonstrates significant innovation, particularly in the depiction of the Kala head and the figures within the Makara's mouth. This research concludes that the animal and mythological iconography of Candi Bumi Ayu is a sophisticated visual text that reveals a complex interplay of religious cosmology, ritual practice, and the assertion of a unique regional identity.

This study investigates how moral values are represented in the Bahasa Indonesia Kawan Seiring textbook for Grade 3 primary schools, published by the Indonesian Ministry of Education, Culture, Research, and Technology (2022). Beyond its role in literacy development, the textbook functions as a curricular artefact through which children internalize moral and civic values aligned with Indonesia’s Profil Pelajar Pancasila. The study employs a qualitative multimodal critical discourse analysis (MCDA), integrating Martin and White’s (2005) Appraisal framework with Kress and van Leeuwen’s (2006) Visual Grammar. The corpus consists of eight textbook chapters containing dialogues, narratives, teacher prompts, illustrations, and reflective activities. Segments with moral orientations were identified, coded into value categories (e.g., helpfulness, politeness, tolerance, responsibility, creativity, diversity), and tabulated. Verbal texts were examined for evaluative stance (affect, judgment, appreciation), while visual texts were analyzed for ideational, interpersonal, and textual meanings. Each value was then mapped against the six dimensions of the Profil Pelajar Pancasila. Findings show that the most salient values are helpfulness, politeness, and tolerance, followed by responsibility, respect, and creativity. These values are linguistically realized through softened imperatives Apakah kamu perlu bantuanku?, affective expressions of empathy, and visually through egalitarian compositions that emphasize cooperation and inclusion. By situating morality in everyday contexts—traditional games, classroom activities, family collaboration, and digital practices—the textbook presents values as lived experiences rather than abstract doctrines. The study contributes to scholarship on textbook-mediated moral education in non-Western contexts and provides practical insights for teachers and policymakers. Kawan Seiring illustrates a hybrid pedagogy that integrates traditional norms with emerging priorities such as digital citizenship and global mindedness, positioning children as both moral agents and future global citizens.

ABSTRACT This article examines how images and hashtags circulated on Twitter during the Tigray War (2020–2022) functioned as hyperreal simulations of nationalism. Drawing on Jean Baudrillard's theory of hyperreality and employing a qualitative visual discourse analysis of 30 curated images, the study investigates how digital artefacts displaced empirical realities with aestheticised narratives of legitimacy, victimhood and resistance. By integrating insights from visual culture, digital nationalism and affect theory, the analysis reveals how these visual texts mobilised affective publics and constructed simulated national imaginaries. The study argues that in the digital spectacle of war, nationalism is not merely expressed but fabricated through stylised and emotionally charged representations. These findings contribute to sociological debates on nationalism, conflict and media by demonstrating how hyperreal aesthetics and platform dynamics converge to reconfigure political meaning in the posttruth era.