Multimodal Conversation Analysis Research Articles

Material Constraints for Assistant-Supported Learning

This paper explores classroom desk interaction where the student has a visual impairment (VIS), and the interaction involves a third supportive party, the student’s learning support assistant. Based on video recordings and multimodal conversation analysis, the paper examines how a VIS, his assistant, and the teacher within a contingent socio-material environment work toward solving an assignment. The analysis is organized following the sequential unfolding of the assignment-solving situation, going from a) determining the need for teacher assistance, b) the recruitment of the teacher’s assistance with the assignment, c) how the participation framework for the joint activity of reviewing the assignment is established with the assistant positioning herself as a fellow “learner”, and d) how the issue is identified and solved. The analysis shows the situated properties of the socio-material environment in which the participants and the local material contingencies are assembled and thus become consequential for the collaborative and observable production of the situation.

Playful framings of social robots in dementia care: reconsidering the principle of transparency in interactions with robot animals

Abstract Research on social robots in dementia care has focused on their effects, for example in relation to the patients’ wellbeing or the care-givers’ working environment. Such approaches to social robots treat them as stable objects with a singular function. Combining social gerontology with social studies of science, the current study offers a new angle by asking: How do patients and care-givers in care homes for older people establish a shared definition of the situation in interactions involving robot animals? Drawing on ethnography and multimodal conversation analysis of 211 minutes of video recordings in two care homes in Sweden, we demonstrate the embodied work by which participants in interactions establish activities with robot animals. In contrast to the ideal of transparency in social robotics, we show that a central affordance of the robots is their vagueness, which allows for their inclusion in playful interactions. Playful framings of the robots highlight their social functions and downplay care-giver–patient asymmetries. However, situations where patients resist a playful frame actualise a dilemma of social inclusion, on the one hand, and the right to not participate in play, on the other. Showing this, the article contributes to knowledge on how people age with technology; in particular, it draws attention to the limits of an ideal of transparency when social robots are included in dementia care.

How to engage

This paper studies how children initiate telling-on sequences in kindergartens. The focus is on multimodal practices that children (6–7-year-olds) employ when engaging adults (teachers/caregivers) and peers in the telling-on sequences. The paper demonstrates how children utilise varying multimodal resources as well as display their moral stances in their telling-ons to recruit the adults’ intervention in the conflicts. The analyses also illustrate how the varying participation frameworks co-constructed by the participants have the potential to influence the organisation of the telling-ons and overall conflict resolution. The findings are based on video-recorded naturally occurring interactions in an English-speaking kindergarten in Finland. The data are analysed using the principles of multimodal conversation analysis.

What is in a look? The accountability of gaze in trajectories to conflict.

This study investigates the role of gaze in initiating episodes of conflict by examining, using multimodal conversation analysis, a set of cases in which a recipient is prompted to speak by another's extended gaze. In these cases, this recipient response may be, e.g., "What," or a more elaborate demand for an account, such as "Why are you looking at me like that for?" Here we investigate the characteristics of the gaze that prompts such responses, and what actions such responses constitute. While "What" compositionally resembles other-initiated repair, its sequential position characterizes it as a so-called "go-ahead" action. In these cases, the sequential positioning of such gazes, constituting it structurally as a so-called "pre," alongside its durational characteristics and facial expression, are examined to identify the normative associations of gaze and subsequent conduct that make such gazes accountable.

Maximizing students’ content and language development: The pedagogical potential of translanguaging in a Chinese immersion setting

BackgroundRecent studies highlight the benefits of translanguaging for emergent bi/multilinguals (EMLs), enhancing their understanding of academic content in a new language. Despite the growth of Chinese immersion programs in the U.S., research on translanguaging and its impact on teacher and student experiences in this context remains scarce. AimsThis study aims to explore how Content and Language Integrated Learning (CLIL) teachers implement pedagogical translanguaging to support students’ learning in academic content and language within a Chinese immersion context in the U.S., and to understand the perspectives of teachers and students on translanguaging. SampleParticipants were two Chinese CLIL teachers and six of their students from a Chinese immersion school in the U.S. MethodsUsing a participatory design research methodology that combines ethnographic methods with multimodal conversation analysis, we documented the use of pedagogical translanguaging and triangulated data from field notes, classroom artifacts, and interviews. ResultsFindings reveal teachers utilized diverse communicative resources and exercised translanguaging shifts to enhance students’ grasp of advanced vocabulary and academic concepts, facilitating complex idea articulation in Chinese. Both teachers and students valued these strategies for deepening subject understanding and supporting their learning of the Chinese language. ConclusionsThe study highlights the importance of continuous professional development for CLIL teachers to effectively implement pedagogical translanguaging, supporting students’ learning in an immersion setting.

Mitigating ethnic minority students’ Foreign language anxiety through Co-learning in Chinese as an additional language classrooms: A transpositioning perspective

Recent studies on ethnic minority students in Hong Kong have highlighted the difficulties they face when learning Chinese-as-an-Additional-Language (CAL). These linguistic challenges can cause anxiety and hinder their integration into Hong Kong society. Transpositioning is an emerging notion in the field of Applied Linguistics and involves individuals stepping out of their assigned roles and adopting different perspectives through translanguaging. It has been argued that engaging in transpositioning can potentially engage students’ learning and participation in classroom interaction. However, there is limited research on how multilingual classroom teachers can suspend their social roles to alleviate students' anxiety while learning a second language in classrooms. This paper seeks to fill this research gap by examining how a Chinese teacher creates a safe translanguaging space for co-learning, which allows him to move beyond his predefined social roles and alleviate ethnic minority students' anxiety in learning CAL. The study uses Multimodal Conversation Analysis to analyse classroom interaction data. The fine-grained classroom analysis is triangulated with the video-stimulated-recall-interviews with the teacher and students which are analysed using Interpretative Phenomenological Analysis. The paper argues for a Transpositioning-Translanguaging-Co-Learning approach to reduce students' anxiety related to CAL learning. By positioning both the teacher and students as equal contributors to knowledge, this can foster student engagement and active participation in classroom interactions.

"Oh my Dave, your face says it all!": Exploring the Multimodal Practices of Tasting in Live Streaming

Research on the activity of tasting, examined through the lens of conversation analysis, reveals that in face-to-face contexts, tasting is an interactive and sequential process that combines individual sensory experience with a public, witnessable, accountable, and intersubjective dimension. This perspective can be extended to the realm of live-streamed tasting, where streamers demonstrate and communicate the taste of food products to online audiences in real time. By adopting multimodal conversation analysis to scrutinize the unfolding sequence moment by moment, my study aims to demonstrate 1) Three practices to achieve the configuration of "tasting heads," wherein the current taster's face is displayed on-screen. 2) Patterns of gaze withdrawal from the screen and subsequent gaze back to the screen during the tasting process. 3) The performative and animated facial expressions. 4) The structure of responses from both streamers and viewers after the tasting.

Translanguaging for the construction of instructional immediacy in a Mandarin–Japanese crosslinguistic class

Abstract Accumulating studies on teachers’ instructional immediacy consistently show a strong positive correlation with teaching effectiveness. To expand the research in a bilingual and cross-cultural context, the current study investigates the construction of instructional immediacy in a natural classroom setting by theoretically anchoring on translanguaging, which refers to the idea of promoting the integration of all linguistic and nonlinguistic resources to achieve effective pedagogy with multilingual speakers. We quantified the nonverbal facial expressions to compare the teacher’s endeavours in different language modes when constructing instructional immediacy, and we used Multimodal Conversation Analysis (MCA) to examine his endeavours. The results revealed language-specific facial expressions which are attributed to varied teaching purposes. We conclude that translanguaging can serve as an effective tool for fostering instructional immediacy in multilingual classrooms.

Haptic Resources in Pain Communication: New Amputees Redirecting Doctors’ Professional Touch at the Prosthetic Clinic

ABSTRACT Pain assessment is key in deciding whether amputee patients are fit to receive a prosthesis, but its interactional accomplishment is still underexplored. This study adopts multimodal conversation analysis to investigate how pain assessment is carried out during medical visits at an Italian prosthetic clinic involving 77 patients and 24 health professionals. In the analyzed data, doctors carry out pain assessment by asking patients whether they feel pain during palpation of the stump, and patients respond by elaborating on their sensations and using touch in turn. The analysis focuses on a collection of 10 cases in which patients use response expansions and self-touch to reorient doctors’ ongoing inquiry by specifying the type of sensation they experience and its location or to correct doctors’ previous inquiry. The analysis illuminates how the patient’s body becomes a resource for tactile practices that are shared between doctors and patients in the service of a common understanding of patients’ pain. This analysis provides new knowledge of a practice patients use to redirect doctors’ attention and understandings.

Transpositioning in English medium instruction classroom discourse: insights from a translanguaging perspective

The concept of ‘transpositioning’ is an emerging notion in the field of Applied Linguistics, influenced by positioning theory and liquid modernity. Transpositioning involves individuals breaking away from their predefined roles and adopting different perspectives through translanguaging. However, there is currently a lack of research exploring how multilingual classroom teachers can transcend their positions using translanguaging to achieve their specific communication goals in multilingual classrooms. This study is the first study that investigates how an English-Medium-Instruction teacher undergoes transpositioning by adopting different positionings to construct an imaginary scenario and engaging in co-learning with students. The study employs Multimodal Conversation Analysis to analyse the classroom interaction data, and the findings are triangulated with video-stimulated-recall-interview data, which is analysed using Interpretative Phenomenological Analysis. The paper argues that transpositioning is a fundamental aspect of the teacher’s classroom translanguaging competence. This competence involves not only the strategic utilization of multilingual and multimodal resources to construct new knowledge and achieve specific pedagogical goals, but also requires the teacher to selectively draw from his repertoire and modify his identity positions in response to the evolving dynamics of classroom interaction.

The development of an ESL teacher's ability in constructing a virtual translanguaging space in synchronous online language tutorials

The COVID-19 pandemic has rapidly increased the use of synchronous online teaching for English-as-a-second-language (ESL) courses. However, there is a scarcity of research on the necessary teaching abilities for online ESL instruction and the advantages of real-time technologies in supporting students' L2 learning. This virtual ethnographic research examines how an ESL teacher utilizes diverse multilingual and multimodal resources to facilitate students' L2 English learning in real-time online tutorials and develops his translanguaging competence over a 10-week period. The analysis, carried out using Multimodal Conversation Analysis and Interpretative Phenomenological Analysis, reveals that the teacher progressively exhibits his capacity to use a variety of multilingual and multimodal tools and to employ different funds of knowledge to support students' English language learning. Consequently, we emphasize the significance of strengthening online language instructors' pedagogical comprehension of translanguaging to enhance their e-Classroom Interactional Competence.

The embodied nature of students’ engagement and participation during a total physical response activity

ABSTRACT This study examines the embodied nature of students’ engagement and participation during a total physical response activity (TPR) in a beginning ESL classroom. A video excerpt is transcribed. Using multimodal conversation analysis, we examine students’ embodied actions following the teacher’s demonstrations for the verb ‘pick up’, following the teacher’s commands for the verbs ‘pick up’ and ‘drop’, and when the teacher is writing on the board. We found the followings: a) even when students are engaged and participate, they display various levels of engagement and participation, b) students’ engagement and participation at any particular moment can be characterised into one of the four quadrants in Jacknick’s (2021) model, namely engaged participation, engaged non-participation, disengaged non-participation, and disengaged participation, and c) students’ engagement and participation vary not only across students but also within individual students. The study sheds light on the embodied nature of students’ engagement and participation from a multimodal multi-activity perspective, and contributes to the still under-investigated research of students’ embodied engagement and participation in teacher-centred interaction.

A Touch of Companionship

Dance activities are promoted with a range of benefits in institutional settings for people living with dementia (PLWD) and necessitate the active involvement of companions for engagement. This ethnomethodological study uses multimodal conversation analysis to explore how the role of the companion emerges to engage PLWD in dance activities, with special attention on different uses of touch to support engagement in embodied interactions. Analyses of video recordings focus on interactions between companions and PLWD in both structured and unstructured dance activities in institutional settings in Canada, Finland, and the UK. We demonstrate how companions use touch and other multimodal resources to engage participants with others in dance activities. We argue that the role of the companion emerges in interaction and can shift from participant to participant, including PLWD. We conclude with discussion and recommendations for supporting the engagement of PLWD with sensitivity to how actions obligate responses.

Infant embodied requests and teacher-practitioner offers during early childhood education mealtimes

The importance of building healthy relationships with food in children's early years is of paramount importance. Building on prior work exploring the social and linguistic practices in infant eating interaction experiences, this research uses a multimodal conversation analysis approach to explore how mealtime interactions are managed as a co-constructed activity between infants (0–2 years) and early childhood teacher-practitioners. Here we will explore video data recorded during mealtimes in an early childhood setting in Mid-Wales, where infants orient to recruitments for assistance and teachers provide offers of help with food items throughout the data. Analysis demonstrates 1) infant recruitment of help through embodied ‘showing’ an item causing a problem in multimodal ways, initiating joint attention that mobilises an offer from an adult in the shape of ‘do you want me to X’ and 2) adult initiation of an offer of help in the shape of ‘would you like me to X’ that are not prompted by infants ‘showing’ an item. Such practices demonstrate infant social competence in recruiting assistance through multimodal resources, and adult's noticings that help is required and their initiation of provision of assistance. The detailed exploration into the ways in which mealtimes are a collaboratively achieved experience reveals how infants effectively contribute in resourceful ways, and how teacher-practitioner responses frame mealtimes as co-produced activities.

The Role of Gazing Behaviors in Navigating Paired Role-Play Interactional Competence Assessment Tasks

ABSTRACT This paper deals with an end-of-semester interactional competence (IC) assessment of L2 oral communication skills for higher education students. Despite the growing research interest in L2 IC assessment using paired role-play tasks, the non-verbal actions of the learners in these contexts remain largely unexplored. With this in mind, this study uses multimodal Conversation Analysis to examine gazing at role-cards in orienting to, structuring, and navigating through the assessment situations, and displaying IC by attending to task initiation, asking/giving task-relevant details, eliciting/making/delaying offers, and ending the conversation. The findings bring new insights into multimodal aspects of L2 IC assessment.

Multimodal practices of unpacking and repacking subject-specific knowledge in CLIL physics and chemistry lessons

BackgroundDifferent school subjects have their specific meaning-making practices for building and conveying knowledge. Research drawing on the Semantics dimension of the Legitimation Code Theory has noted the importance of shifting between levels of abstraction and context-dependency in knowledge-building. There is a need to better understand how such shifting between different levels of abstraction is accomplished with multimodal resources in classroom interaction. AimsThis study aims at exploring subject-specific knowledge construction as a form of translanguaging, i.e., as movement between different registers and multimodal resources of meaning-making. DataThe data comes from a Finnish teacher development project aimed at supporting CLIL teachers' professional development. This exploratory study analyses teachers’ knowledge-building practices in two STEM lessons video-recorded in the project, Physics and Chemistry. MethodsThe data is analysed using multimodal conversation analysis and analysis of semantic waves. Analysis focuses on how the teachers engage in unpacking and repacking subject-specific knowledge by talking, gesturing, as well as displaying, handling, and modifying various kinds of multimodal materials and artefacts. ResultsThe teachers were found to use a versatile set of multimodal translanguaging practices for unpacking and repacking. The findings also indicate complexity in semantic waves due to multimodal resources accomplishing simultaneous shifts in semantic gravity and density, with either aligning or diverging functions. ConclusionsThe simultaneous use of different multimodal resources and their potential to serve different functions point to the need to acknowledge the multidimensionality of semantic waves. The multimodal translanguaging approach also has implications for conceptualising subject-specific knowledge-building as inherently multimodal.

Female-to-male (FtM) transgender individuals’ multimodal and interactional practices in Japanese

Abstract This study explores the multimodal and interactional practices of female-to-male (FtM) transgender individuals in Japanese, focusing on their identity construction. Most previous studies on transgender identity in Japanese contexts examine male-to-female (MtF) transgender individuals’ linguistic practices, particularly the use of so-called Japanese women’s language. In contrast, this study explores how FtM individuals use not only linguistic but also other semiotic resources to construct and negotiate their gender and sexual identities. The study utilizes empirical data such as YouTube videos and draws on multimodal conversation analysis (CA) and membership categorization analysis (MCA) to analyze the participants’ accomplishment of social actions. The findings demonstrate the use of various resources for identity construction and how the participants challenge and/or conform to hegemonic gender ideologies. The study also unveils both the diversity among FtM transgender people (inter-individual variation) and the fluidity and complexity of identities that can exist within one person (intra-individual variation).

Book review: Multimodal conversation analysis and interpretative phenomenological analysis: A methodological framework for researching translanguaging in multilingual classrooms

Book review: Multimodal conversation analysis and interpretative phenomenological analysis: A methodological framework for researching translanguaging in multilingual classrooms

Daring to taste: The organisation of children's tasting practices during preschool lunches

Tasting food is the first step toward diversifying eating habits, and studies with children have typically focused on their sensory education and willingness to try new foods. While very little is known about how children taste foods during everyday mealtimes, EMCA (ethnomethodological and conversation analytic) research on adult tasting in naturalistic settings has demonstrated regular organisational patterns. This paper brings these two research areas together, using the insights of EMCA research on adult tasting to inform our understanding of how young children taste food during preschool lunches. Data are taken from a large corpus of video-recorded lunches in Sweden, in which children (3- to 6-year-olds) were eating with staff members. Discursive Psychology and multimodal Conversation Analysis were used to analyse the data. The analysis demonstrates how the sequential organisation of child tasting is similar to adult tasting, and how tasting practices are a collaborative, multisensory activity involving various embodied practices: from the orientation to food as ‘to be tasted', the withdrawal of mutual gaze and exaggerated mouth movements, to the re-establishment of gaze accompanying the food assessment. In contrast to adult tasting, however, tasting during preschool lunches is often framed in terms of personal development of the child and of the individualising of taste within the framework of the institution. The findings thus provide further support for EMCA research on sensory practices and contribute to psychological research on children's eating by evidencing the importance of the interactional and institutional context on tasting as a sensory practice.

Discourse-related code-switching in conversations with a bilingual autistic adult

Background: Although studies on language alternation among non-autistic bilinguals have increased in the last decades, we know little about this linguistic behavior among autistic bilinguals. Research on this topic is particularly relevant, given its potential to further our understanding of the ability of autistic bilinguals to deploy language alternation discursively and orient to different aspects of ongoing conversation. Method: We used multimodal conversation analysis to analyze 237 instances of language alternation embedded in naturalistic conversations between a bilingual autistic adult with severe language impairment and his frequent co-interactants. Specifically, 157 of those language alternative instances served conversational functions (e.g., setting off quotes, contextualizing addressee change, highlighting repair execution, adding emphasis) and were therefore associated with discourse-related code-switching. Discussion/conclusion: The study’s findings highlight the need for the different stakeholders to critically ponder their assumptions about the ability of autistic individuals to become bilingual; to resist a tendency to default to monolingualism in the presence of an autism diagnosis; and to (re)consider their beliefs about the nature of language alternation.