Articles published on Multimodal Communication
Authors
Select Authors
Journals
Select Journals
Duration
Select Duration
1251 Search results
Sort by Recency
- New
- Research Article
- 10.1002/msc.70219
- Jun 1, 2026
- Musculoskeletal care
- Sini Puustinen + 2 more
The therapeutic relationship is meaningful and valuable to patients in rehabilitation and is positively associated with improved rehabilitation outcomes. Despite its central role, a comprehensive understanding of what constitutes the therapeutic relationship in physiotherapy and how it can be assessed is yet to be achieved. This study aimed to identify the core components of the therapeutic relationship and the patient-reported instruments used to assess it in musculoskeletal physiotherapy. A systematic integrative review was conducted. Six databases (CINAHL, PubMed/Medline, Scopus, Web of Science, The Cochrane Library, and PEDro) were searched. Two reviewers independently conducted study selection and methodological quality appraisal. Data were synthesised using inductive qualitative content analysis. Eighteen studies were included. Two core components and six subcomponents of the therapeutic relationship in musculoskeletal physiotherapy were identified: (1) therapeutic partnership (partnership and therapeutic communication) and (2) collaborative person-centred physiotherapy (collaboration; a holistic and individualised approach; coherent, competent, and credible physiotherapy; and empowerment support). Six instruments used to assess therapeutic relationships in musculoskeletal physiotherapy were identified. In musculoskeletal physiotherapy, the therapeutic relationship is a multidimensional and dynamic construct encompassing therapeutic partnership and collaborative person-centred practice. Physiotherapists can actively influence the therapeutic relationship by utilising multimodal communication, implementing person-centred care strategies, and supporting patients' agency and empowerment. Existing patient-reported therapeutic relationship instruments capture these components only partially. Future research is needed to strengthen conceptual clarity and refine both the operationalisation and assessment of the therapeutic relationship in musculoskeletal physiotherapy.
- New
- Research Article
- 10.1080/02687038.2026.2663794
- May 14, 2026
- Aphasiology
- Sarah Northcott + 6 more
ABSTRACT Background and aim Telehealth is increasingly used within healthcare, including by Speech and Language Therapists working with people who have aphasia. Aphasia is a communication disability common post stroke that presents challenges for telehealth, (e.g. people with aphasia are more dependent on multimodal communication techniques), which can be difficult to interpret through videoconferencing platforms. Assessments are a core part of the SLT role, and can pose particular challenges for telehealth, (e.g. testing receptive language skills). This study explored how people with aphasia and clinicians experience telehealth in the context of completing assessments, including their priorities for future directions. Methods and procedures This study integrated four different research activities from the “Improving access to healthcare: Face-to-face And Telehealth Equivalence of assessments in Aphasia (FATE-A)” study. The four data sources used were: qualitative responses to a survey (n = 124 Speech and Language Therapists), a focus group study (n = 14 Speech and Language Therapists), usability testing sessions (n = 4 Speech and Language Therapists, n = 6 advisers with aphasia) and Patient and Public Involvement and Engagement (PPIE) workshops (n = 7 advisers with aphasia). Findings from the four sources were inductively analysed to develop descriptive themes taking a meta-synthesis approach. Divergence and convergence of themes between sources was explored. Outcomes and Results There were three main themes: barriers, facilitators, and current and future preferences. Barriers and facilitators included severity of aphasia and other impairments; presence or absence of helpers; competence with technology (both clinician and person with aphasia); challenges around hardware and video-conferencing platforms; poor versus reliable internet connectivity; factors specific to assessments (e.g. lack of standardized assessment for online use); challenges setting up online assessments; and the emotional impact. Additional facilitators included preparation and support with set-up, and support with emotional wellbeing. Clinicians wanted more resources to facilitate online assessment, better hardware and platforms. While clinicians and people with aphasia perceived benefits to telehealth, both stakeholder groups wanted a flexible personalized hybrid service. Although there was broad congruence of themes, different data sources contributed uniquely, for example, the challenges of negotiating multiple steps to get online (usability testing), and preferred terminology (PPIE). Conclusions It is likely that telehealth will continue to be a component of Speech and Language Therapy services. Both people with aphasia and clinicians considered telehealth a valid model of service delivery, so long as it is tailored to the needs of the individual, and is part of a person-centred model of care.
- Research Article
- 10.1016/j.xgen.2026.101141
- May 13, 2026
- Cell genomics
- Chenfeng Mo + 2 more
MultiSP deciphers tissue structure and multicellular communication from spatial multi-omics data.
- Research Article
- 10.2460/javma.26.01.0077
- May 6, 2026
- Journal of the American Veterinary Medical Association
- Janice S O'Brien + 5 more
To quantitatively assess veterinarian perceptions of barriers and solutions to pet nutrition communication during small animal appointments in the US and Canada. An anonymous online survey was circulated by the Veterinary Information Network to its members between March 28 and April 13, 2022. Respondents included veterinarians in active practice. The survey included fixed-choice and free-text response items. Fixed-choice items were tabulated with R software. Free-text items were coded thematically with Atlas.ti software. From 561 survey respondents, the top 3 barriers identified were pet owner preconceived notions about nutrition (86% [483 of 561]), insufficient time (63% [352 of 561]), and pet owner resistance to discussing nutrition (41% [230 of 561]). The top 3 solutions identified were showing/talking to owners about what veterinary professionals feed their pets (59% [332 of 561]), having direct yet compassionate conversations with owners (58% [327 of 561]), and specific nutrition recommendations (49% [274 of 561]). Free-text responses clarified nuances within each potential solution, such as the importance of establishing veterinary-owner trust first and not being too technical with specific recommendations. Small animal veterinarians reported that pet owner preconceived notions about nutrition were the most common reported barrier to nutrition communication. What veterinary professionals feed their pets, direct yet compassionate conversations, and specific nutrition recommendations were reported as potential solutions, with the caveat that veterinary-owner trust was important to establish first. Building trust with pet owners, being proactive with direct yet compassionate nutrition conversations, and including specific nutrition recommendations may be helpful in breaking down barriers to nutrition communication.
- Research Article
- 10.1080/10447318.2026.2664700
- May 6, 2026
- International Journal of Human–Computer Interaction
- Huaizhuo Yang + 4 more
To increase public acceptance of autonomous vehicles (AVs), HCI researchers have explored interaction designs that foster user trust. This article introduces a novel concept: designing AV interfaces to mimic the social presence and behavioral traits of an experienced, attentive human driver, leveraging people’s greater trust in such drivers over opaque automation. To investigate this idea, we conducted two co-design workshops. Using the Annotated Portfolio method, we derived design annotations from visual artifacts (e.g., prototype photos and sketches), which were analyzed alongside workshop transcripts through Reflexive Thematic Analysis (RTA). We present design guidelines for AV interfaces that embody a trustworthy human driver, including: (1) proactive multimodal communication of situational awareness; (2) use of a driving agent (robotic, virtual, vocal, or spatial); (3) gradual development of human–AV companionship; and (4) personalization to foster familiarity and trust. Methodologically, we demonstrate integrating Annotated Portfolios and RTA to generate design insights.
- Research Article
- 10.1186/s40900-026-00900-w
- May 4, 2026
- Research involvement and engagement
- Ravi Patel + 10 more
Patient and Public Involvement and Engagement (PPIE) is integral to ensuring health research remains grounded in patient priorities and lived experience. This is especially critical for underrepresented groups, such as adults aged 90 and over considering elective total hip replacement (THR). This demographic experiences a small but meaningful risk of postoperative mortality, yet conventional numeric risk communication often fails to align with their values, communication preferences, and the existential context of very advanced age. Traditional PPIE methods, like focus groups, may inadvertently exclude this group due to sensory, mobility, or digital barriers. This PPIE activity engaged eight public contributors aged 90-96 years with lived experience of total hip replacement or caring for someone who underwent it (including two carers) through semi-structured one-to-one telephone conversations, aligned with the GRIPP2 Short Form and UK Standards for Public Involvement. We detail our approach to building rapport and facilitating nuanced conversations remotely, focusing on three domains: research context, communication preferences, and decision-making. Our methodology was designed to prioritise relationship-building and flexibility to overcome barriers to participation. Inductive review of facilitator notes identified three central themes: 1.A Pragmatic Conceptualisation of Risk: Participants framed mortality risk as an acceptable "chance you take," shaped by life experience and advanced age.2.Values-Based Decision-Making: Quality of life outcomes: mobility, independence, and maintaining an optimistic outlook were prioritised over longevity alone.3.The Imperative for Compassionate Communication: Participants emphasised a need for clear, respectful, and multi-modal communication, while explicitly rejecting ageist or overly clinical language. Contributors unanimously affirmed the importance of this topic, feeling that as an underserved community, their perspectives on mortality were both valuable in decision making and long overdue. This PPIE activity involved a small number of contributors and used telephone-only engagement. While this ensured accessibility, it may limit transferability to other contexts. Future work should test alternative formats and include larger, more diverse samples to enhance generalisability to the nonagenarian populations. Nonetheless, the rich feedback insights from this unrepresented group provide a valuable foundation for future work. The findings challenge the primacy of numerical risk presentation for this demographic, underscoring that communication must prioritise dignity, optimism, and clarity. Crucially, contributors affirmed that discussing mortality is a necessary part of informed consent, countering assumptions that this topic should be avoided. These insights directly inform more sensitive and relevant research design and clinical communication tools, ensuring they are grounded in the values and priorities of the patients they are meant to serve.
- Research Article
- 10.56773/ierj.v3i3.118
- Apr 30, 2026
- Indonesian Educational Research Journal
- Abdulwasiu Isiaq Nasirudeen + 1 more
Given the rapid integration of digital technologies in language education and the unique linguistic complexity of Arabic, urgent research is needed to empirically determine how learners’ digital literacy competencies influence Arabic as a Foreign Language acquisition. This mixed-methods study investigates the relationship between digital literacy levels and language proficiency among 124 non-native Arabic learners enrolled in university-level AFL programs. Quantitative data were collected using a validated digital literacy scale adapted from the European Digital Competence Framework (DigComp) and a standardized Arabic proficiency test aligned with ACTFL benchmarks. Qualitative interviews and classroom observations explored learner perceptions and instructional practices. Results revealed statistically significant correlations between digital literacy skills-particularly information navigation, multimodal communication, and content creation-and reading comprehension, writing accuracy, and listening proficiency. Findings indicate that digitally literate learners demonstrate greater autonomy, increased exposure to authentic media, and more effective utilization of learning tools. Key challenges include unequal access to technology, limited pedagogical integration of digital tools, and learner dependency on machine translation. The study underscores the necessity of embedding digital literacy instruction within AFL curricula and provides recommendations for teacher training, curricular design, and future research
- Research Article
- 10.59324/ejaset.2026.4(3).03
- Apr 28, 2026
- European Journal of Applied Science, Engineering and Technology
- Haseeb Ur Rehman + 5 more
Semantic communication is a potential promising paradigm in next-generation communication systems, in which the emphasis is laid on the transfer of meaningful information, as opposed to bare data. Although it has the potential to enhance efficiency of communication and exchange of information intelligently, semantic distortion due to noise channels is a significant threat, especially in multi-modal communication situations. This paper presents a multi-modal semantic communication system based on deep learning that features fully automated error detection and correction. The framework shares visual and textual information with distinct encoders of image and text and combines their semantic representations in a common latent space and sends the combined representation across a noisy communication channel. The semantic representation sent is decoded to recreate the image at the receiver side and re-extract the equivalent textual information. To enhance resilience, a loss-based semantic error detection system is suggested to detect damaged transmissions and an automated correction plan is implemented each time the detected error is more than a pre-determined threshold. The suggested framework is tested on several benchmark data sets, such as MNIST, FashionMNIST, CIFAR-10, and EMNIST, to test its ability to withstand the needs of various data distributions. The experimental findings show that the proposed model can satisfy stable multi-modal semantic recovery and effective semantic error detection in all the tested datasets, and the correction mechanism offers some, but not significant, quantitative advantages in noisy environments. The framework proposed provides a dynamic and practical solution to intelligent multi-modal semantic communication in future communication networks like beyond-5G and 6G networks.
- Research Article
- 10.9734/ajess/2026/v52i42992
- Apr 28, 2026
- Asian Journal of Education and Social Studies
- Glorida Nachimma-Lopez
Social media platforms like Facebook have become key spaces for dynamic, multimodal communication, where users combine text, images, emojis, and other resources to construct meaning. In Filipino digital discourse, these practices reflect rich linguistic creativity and relational work, shaping how identity, social interaction, and meaning are negotiated online. This study examines the multimodal communicative practices of Filipino Facebook users, focusing on how linguistic and non-linguistic resources interact to construct meaning in computer-mediated discourse. Drawing on a corpus of 315 anonymized expressions from publicly accessible Facebook posts, the study employs a qualitative discourse-analytic approach grounded in multimodal discourse analysis and relational work theory. Methodological rigor is enhanced through brief interpretive interviews for communicative validation and expert linguistic review by two Filipino language scholars. The analysis identifies key linguistic features, including word-formation processes (blending, clipping, affixation, reduplication), sound-based expressions (interjections, letter elongation, onomatopoeia), non-standard orthography, and creative colloquial forms such as reverse spelling (sakalam from malakas). These linguistic resources co-occur with non-linguistic elements such as emojis, emoticons, GIFs, stickers, and images. Together, these multimodal resources perform expressive, phatic, humorous, and affiliative functions that support emotional clarity, relational work, and interactional efficiency in text-based digital environments. The findings demonstrate how Filipino users mobilize culturally specific multimodal repertoires, exemplified by emoji-anchored code-mixing (e.g., Favorite ko yan) and affectively charged respellings, to manage affect and social alignment in everyday digital interaction. By focusing on Filipino Facebook discourse, this study contributes to computer-mediated communication research by highlighting the culturally situated nature of multimodal meaning-making and challenging Anglocentric assumptions in digital discourse studies. The findings also offer implications for digital literacy education in multilingual Southeast Asian contexts.
- Research Article
- 10.1126/sciadv.aee2625
- Apr 24, 2026
- Science advances
- Muzi Xu + 9 more
The human skin acts as a dynamic biomechanical interface that conveys critical physiological and behavioral information through spatiotemporally distributed deformations. Because of the limited capabilities of current sensing technologies, the spatiotemporal diversity of its mechanical cues has remained underused to date, preventing these mechanisms from being used to capture and decode the full spectrum of underlying physiological states. In this work, we define this heterogeneous set of mechanical signals as mechanodermal activity (MDA) and introduce the biomimetic metamaterial-based interface (BMMI), an engineered auxetic metamaterial substrate that reproduces the microrelief and mechanoreceptor architecture of natural skin. The BMMI allows selective capture of diverse MDA signals from adjacent skin regions with simultaneous signal amplification and noise suppression and permits straightforward modulation to accommodate various scenarios. Combined with bespoke algorithms, the wireless BMMI device decodes MDA accurately and robustly for multimodal communication interfaces, unleashing applications in health care monitoring and human-machine interaction.
- Research Article
- 10.1002/adfm.75589
- Apr 23, 2026
- Advanced Functional Materials
- Shuwen Yuan + 11 more
ABSTRACT Next‐generation intelligent perception systems require photodetectors that are functionally reconfigurable for in‐sensor processing. However, integrating broadband response, polarization sensitivity, and electrically reconfigurable photoresponse into a single compact device remains challenging. Here, we demonstrate a solution using a gate‐programmable photodetector based on a 2H‐MoTe 2 /ReSe 2 heterojunction. The device exhibits self‐powered broadband detection from 300 to 900 nm, with a responsivity of 0.945 A/W, a specific detectivity of 1.41 × 10 12 Jones, and a polarization ratio of 16.45 at 635 nm. Critically, gate voltage asymmetrically modulates the dual built‐in fields via Fermi‐level tuning, enabling continuous and programmatic reconfiguration of the magnitude and polarity of photocurrent. The responsivity is tuned from 0.142 to −1.254 A/W, and this polarity reversal allows the polarization ratio range to span both positive and negative domains. Leveraging this unique programmability, we realize two advanced applications: multimodal encryption communication, where gate voltage and polarization angle serve as dual dynamic encryption keys, and in‐sensor image convolution for direct edge extraction at the sensor level. This work establishes a robust platform toward multifunctional and reconfigurable optoelectronics, advancing the development of all‐in‐one intelligent perception.
- Research Article
- 10.64751/b7cd7a12
- Apr 21, 2026
- International Journal of AI Electronics and Nexus Energy
- S Sreenath Kashyap + 5 more
The rapid evolution of underwater communication technologies has become essential for applications such as marine exploration, environmental monitoring, and defense systems. Historically, underwater communication has relied primarily on acoustic methods due to their ability to travel long distances through water. However, with the increasing demand for high-speed and reliable data transmission, traditional systems face significant challenges. Acoustic communication, while effective over long ranges, suffers from low data rates, high latency, and susceptibility to noise and signal distortion. Similarly, radio frequency (RF) communication is highly inefficient underwater due to severe attenuation. These limitations highlight the need for an advanced communication approach capable of providing faster and more efficient data transfer in underwater environments. In this context, Light Fidelity (LiFi) technology emerges as a promising solution by utilizing visible light for data transmission. The proposed system integrates LiFi for high-speed, short-range underwater data communication and acoustic communication for long-range transmission, thereby combining the advantages of both technologies. The system employs LED-based transmitters and photodetector receivers for optical communication, along with acoustic modules for extended reach. This hybrid approach enhances data transmission efficiency, reduces latency, and improves reliability in dynamic underwater conditions. The significance of this work lies in its ability to overcome the inherent limitations of conventional systems, offering a scalable and efficient solution for nextgeneration underwater communication devices, with potential applications in oceanographic research, underwater robotics, and naval operations.
- Research Article
- 10.54691/0kczfy60
- Apr 20, 2026
- Frontiers in Humanities and Social Sciences
- Bin Liu
Against the backdrop of the deep integration of globalization and digitalization, the international communication of Huxiang culture faces multiple challenges including "cultural discount," platform dependence, and stereotyping. Based on multimodal discourse analysis theory and digital humanities technologies, and drawing on Hunan Province's communication practices of "integration of culture and technology" and "integration of culture and tourism," this study constructs multimodal communication pathways for Huxiang culture encompassing technological empowerment and communication strategies, with the aim of providing theoretical frameworks and practical insights for effective international communication of Huxiang culture.
- Research Article
- 10.34118/jskp.v6i1.4552
- Apr 17, 2026
- Journal of Science and Knowledge Horizons
- Nasir Razzaq + 2 more
Digital communication has reshaped how multilingual speakers make pragmatic language choices in everyday interaction. Digital platforms reorganize language through speed, multimodality, and technical affordances in emerging multilingual societies. As undergraduate students are intensive users of digital media, their interactional practices offer a key site for examining pragmatic choices across face-to-face and online contexts. This study examines how Tanzanian and Pakistani undergraduate students negotiate language mixing and language purity, with particular attention to digitally mediated interaction. Through observation, semi-structured interviews, and focus group discussions, qualitative, inductive, and comparative data were collected and analyzed thematically. The findings indicate that Pakistani students rely heavily on Romanised Urdu for digital convenience, while experiencing increasing difficulty with Urdu script and vocabulary, suggesting script attrition and lexical erosion. Emojis and memes provide multimodal pragmatic communication of humour, attitude, tone, and emotion. Habitual code-mixing reflects lexical automatisation, alongside participant concern regarding younger generations’ diminishing lexical depth in local languages. The study concludes that pragmatic choice extends beyond speech to include scripts, visual modes, and digital infrastructures, with implications for language maintenance in digital futures
- Research Article
- 10.58218/alinea.v6i1.2386
- Apr 15, 2026
- Alinea: Jurnal Bahasa, Sastra dan Pengajaran
- Nurfidah Nurfidah + 3 more
The transformation of digital business communication has shifted persuasive practices from linear textual approaches toward multimodal, interactive, and data-driven communication. However, business persuasive writing instruction in higher education remains largely conventional, limiting the development of multimodal competence, audience analysis, and data utilization. This study aims to identify the gap in the integration of digital rhetoric in business persuasive writing instruction at Lombok Business Academy (AKBIL). This study employed a mixed-methods approach with a sequential explanatory design, involving a survey of 80 students, analysis of written documents, and interviews with students and lecturers. Quantitative data were analyzed using descriptive statistics, while qualitative data were examined through thematic analysis to enrich interpretation. The findings indicate that the overall level of digital rhetoric integration is moderate (M = 3.12). Audience awareness shows a relatively higher score (M = 3.34), yet remains general and not supported by data-driven analytics. Multimodal integration (M = 3.05) is not yet strategic, as visual elements are primarily used decoratively. Data-driven persuasion emerges as the weakest dimension (M = 2.84), characterized by limited use of quantitative evidence and data visualization. Qualitative findings further reveal that instruction remains text-oriented, with limited pedagogical readiness among lecturers and low levels of student data literacy. The study highlights a gap between the demands of digital business communication and current pedagogical practices. Therefore, it underscores the need for developing a digital–rhetorical pedagogy model that systematically integrates multiliteracies, data-driven audience analysis, and evidence-based argumentation in business persuasive writing instruction.
- Research Article
- 10.9734/ajess/2026/v52i42968
- Apr 14, 2026
- Asian Journal of Education and Social Studies
- Israa Ismael + 2 more
In the 21st century, English as a Foreign Language (EFL) literacy extends beyond basic reading and writing to encompass a broader set of competencies shaped by digital communication, information abundance, and global connectivity. Learners are increasingly expected to engage critically with information, interpret multimodal texts, and communicate meaning effectively across diverse contexts. Despite this shift, EFL instruction in many contexts, particularly in settings such as Egypt, remains largely skill-based and exam-oriented, emphasizing memorization and surface-level language proficiency. This article adopts a conceptual and analytical perspective to examine the gap between contemporary understandings of literacy and prevailing classroom practices. It critically discusses the limitations of traditional literacy models and the challenges posed by digital learning environments. Drawing on recent literature, the paper proposes pedagogically grounded directions for bridging this gap, including the integration of AI as a tool for critical engagement, practice-based teacher development, assessment approaches that prioritize higher-order thinking, and the incorporation of cognitively demanding tasks into everyday instruction. These shifts aim to reposition EFL classrooms as spaces for critical inquiry, multimodal communication, and informed participation in digitally mediated environments, better aligning language education with the demands of the 21st century.
- Research Article
- 10.1080/1350293x.2026.2653671
- Apr 10, 2026
- European Early Childhood Education Research Journal
- Olivia Karaolis + 1 more
ABSTRACT Children’s right to be heard is central to both the UN Convention on the Rights of the Child and the UN Convention on the Rights of Persons with Disabilities. Yet in many early childhood settings, participation is narrowly defined in ways that may inadvertently marginalise or exclude disabled children. This paper draws on research from an ongoing study, that reimagines participation using portraiture methodology, puppetry, and Lundy’s [2007. “‘Voice’ is not enough: Conceptualising Article 12 of the United Nations Convention on the Rights of the Child.’ British Education Research Journal 33 (6): 927–942] model of participation which addresses space, voice, audience, and influence. Within this paper we explore how puppetry can support children to express themselves through gesture, affect, material interaction, and relational presence. Using the art-form of puppetry as provocation and as a relational scaffold for multimodal communication, children’s ideas shape routines, curriculum, and relational practices. This research contributes to early childhood scholarship by offering concrete, anti-tokenistic approaches to inclusion and communication rights.
- Research Article
- 10.1002/epi.70229
- Apr 3, 2026
- Epilepsia
- Elizabeth M Watson + 9 more
We created composite maps of language function from extraoperative stimulation literature and transformed them to the Yale Brain Atlas (YBA), which offers precise cortical localization with 690 one cm2 parcels, based on the MNI152 template and anatomical landmarks. This allowed comparison to similarly transformed direct cortical stimulation (DCS) maps created from medically intractable epilepsy patients studied intracranially at Yale University and selected fMRI activation data. Our goal was to create anatomically precise boundaries of language function and support individualized planning for intracranial EEG (icEEG) studies and/or surgical resection. A systematic search of DCS studies of language-related functions identified 12 stimulation studies (10 subdural, two stereo-EEG) that provided sufficiently clear surface visualizations for parcel-level mapping. Language-positive sites from six commonly tested functions (visual naming, auditory naming, naming (auditory/visual), reading, repetition and speech arrest) were manually co-registered to YBA parcels using conserved anatomical landmarks, facilitating topological comparison among similar language functions from the literature, Yale's electrical stimulation mapping cohort, and Neurosynth and Parcelsynth fMRI activation. Across 1182 patients, 176 YBA parcels (approximately 25% of cortical parcels) were implicated in at least one language task. Composite stimulation maps demonstrated robust consistency with the Yale DCS cohort and substantial overlap with distributed fMRI language networks. The resulting interactive maps provide parcel-level specificity while highlighting under-sampled but functionally critical regions. The maps provide a review of language function determined through DCS and meta-analytic repositories of fMRI activation data. By consolidating stimulation-derived language data into a high-resolution parcellated atlas, this work offers a spatially detailed surface-based representation of language boundaries for surgical planning. The YBA facilitates standardized multimodal communication and allows patient-specific stimulation results to be interpreted within population-derived functional boundaries.
- Research Article
- 10.1016/j.anbehav.2026.123526
- Apr 1, 2026
- Animal Behaviour
- L Habib-Dassetto + 7 more
Communication is an interactive social act involving the combination of multimodal signals and actions rooted in cognitive and social foundations with potentially deep evolutionary origins. Historically, research on animal communication has focused either on monoagent, unimodal sequence organizations or signal onsets, excluding overlaps and silences, which leads to missing out on potentially relevant communicative information and limits interspecific comparisons. Researchers increasingly call for multimodal and interactive approaches, including the integration of actions alongside communicative signals. However, no tool has been developed to fully process such complex data. In this study, we propose a new processing framework equipped with an open-access tool (‘Multi-interaction’) that aims at simplifying the study of multimodal communicative exchanges and cross-species comparisons. First, this approach considers intraindividual and interindividual overlaps occurring during any communicative sequence by transcribing overlapping units together rather than as separate elements. It also allows the flexible categorization of units, enabling researchers to classify species-specific elements into broader, species-general categories and adapt analyses to any annotated data set, modality or granularity level. Second, this tool can be used to transcribe each annotated multimodal interaction into a unidimensional discrete sequence that preserves the type of units making up the interaction (including silences), their order, emitter and overlap. Third, the proposed approach extracts automatically quantitative variables describing sequence properties, such as the overall duration, number of units, diversity of units, proportion of interindividual or intraindividual overlap or proportion of each individual contribution during the interaction. It also allows researchers to build automatically co-occurrence matrices of units (i.e. unit associations) that can capture intraindividual and interindividual associations either sequentially or in overlaps. To our knowledge, this framework is the first to propose a generic method to process quantitatively multimodal and multiagent information, including overlaps and silences, which will enable the scientific community to handle large data sets and cross-species comparisons. • Comparative studies lack consistent methods for multimodal interaction analysis. • We introduce a unified framework for studying multimodal interactions. • We propose new methods for annotating, processing and analysing interaction data. • Interaction units, emitters, overlaps and silences are preserved throughout. • This approach enables species-general comparisons of interactions.
- Research Article
- 10.1016/j.tele.2026.102390
- Apr 1, 2026
- Telematics and Informatics
- Yiran Du + 5 more
• Examines how emojis influence irony comprehension and memory in computer-mediated communication. • Employs a three-phase recognition memory task with participants from the UK and China. • Finds irony is harder to comprehend but more memorable than literal statements. • Identifies culture-specific effects: smiling emojis aid Chinese participants, winking emojis aid UK participants. • Advances theoretical understanding of irony, memory, and multimodal digital communication. This study investigates how emojis shape the comprehension and memory of verbal irony in computer-mediated communication across UK and Chinese participants. Using a three-phase recognition memory task and a mixed factorial design, the research examines how literality, emoji type, and cultural background influence both real-time interpretative accuracy and later recognition. The findings show that irony is harder to comprehend but easier to remember than literal statements, and that culturally preferred emojis, smiling faces for Chinese readers and winking faces for UK readers, enhance both irony comprehension and mnemonic performance. These effects align with Echoic Mention Theory, Dual Coding Theory, and Levels-of-Processing accounts, suggesting that emojis operate as culturally grounded socio-pragmatic cues that deepen processing and enrich memory traces. Overall, the results highlight the multimodal and culturally sensitive nature of irony processing in computer-mediated communication.