Multiparty Interactions Research Articles

Beyond Binary Dialogues: Research and Development of a Linguistically Nuanced Conversation Design for Social Robots in Group–Robot Interactions

In this paper, we detail the technical development of a conversation design that is sensitive to group dynamics and adaptable, taking into account the subtleties of linguistic variations between dyadic (i.e., one human and one agent) and group interactions in human–robot interaction (HRI) using the German language as a case study. The paper details the implementation of robust person and group detection with YOLOv5m and the expansion of knowledge databases using large language models (LLMs) to create adaptive multi-party interactions (MPIs) (i.e., group–robot interactions (GRIs)). We describe the use of LLMs to generate training data for socially interactive agents including social robots, as well as a self-developed synthesis tool, knowledge expander, to accurately map the diverse needs of different users in public spaces. We also outline the integration of a LLM as a fallback for open-ended questions not covered by our knowledge database, ensuring it can effectively respond to both individuals and groups within the MPI framework.

Is It Possible to Recognize a Speaker Without Listening? Unraveling Conversation Dynamics in Multi-Party Interactions Using Continuous Eye Gaze

Is It Possible to Recognize a Speaker Without Listening? Unraveling Conversation Dynamics in Multi-Party Interactions Using Continuous Eye Gaze

Abstract Continuation Semantics for Multiparty Interactions in Process Calculi based on CCS

Abstract Continuation Semantics for Multiparty Interactions in Process Calculi based on CCS

A Study on Consumers’ Willingness to Purchase Autonomous Vehicles from a Multi-Party Interaction Perspective: A Tripartite Evolutionary Game Model Involving the Government, Automobile Manufacturers, and Consumers

With the rapid development of autonomous driving technology, the advent of the autonomous driving era has become inevitable. An in-depth study of consumers’ willingness to purchase autonomous vehicles is critical to accelerating the adoption and commercialization of autonomous vehicles. By constructing a tripartite evolutionary game model of governments, automobile manufacturers, and consumers, we analyze the stable choice of unilateral strategy and equilibrium strategy of autonomous vehicle purchase intention. The MATLAB2022b tool was used for data simulation analysis to verify the validity of the conclusion and the influence of related factors on the purchase intention toward autonomous vehicles. The results show the following: (1) The combination of government support, active R&D, and consumer purchasing is the evolutionary stability strategy (ESS) of the model. (2) With an increase in government support, the probability of automobile enterprises taking the initiative to participate in R&D also increases. However, the negative impact of risk can significantly reduce the incentive for firms to conduct R&D and reduce the effectiveness of government support. (3) Government subsidies to consumers and purchase incentives offered by automotive companies can significantly increase the likelihood that consumers will purchase an autonomous vehicle. Based on these findings, recommendations are made to strengthen government support, establish risk mitigation mechanisms, and strengthen market promotion efforts to promote the commercialization of autonomous vehicles. The study provides a new perspective for understanding multi-party interactions in the rollout of autonomous vehicles and provides valuable insights for policymakers and industry stakeholders.

HVCMM: A Hybrid-View Abstractive Model of Automatic Summary Generation for Teaching.

With the continuous development of educational informatization, more and more emerging technologies are applied in teaching activities. These technologies provide massive and multidimensional information for teaching research, but at the same time, the information obtained by teachers and students presents an explosive increase. Extracting the core content of the class record text through text summarization technology to generate concise class minutes can significantly improve the efficiency of teachers and students to obtain information. This article proposes a hybrid-view class minutes automatic generation model (HVCMM). The HVCMM model uses a multilevel encoding strategy to encode the long text of the input class records to avoid memory overflow in the calculation after the long text is input into the single-level encoder. The HVCMM model uses the method of coreference resolution and adds role vectors to solve the problem that the excessive number of participants in the class may lead to confusion about the referential logic. Machine learning algorithms are used to analyze the topic and Section of the sentence to capture structural information. We test the HVCMM model on the Chinese class minutes dataset (CCM) and the augmented multiparty interaction (AMI) dataset, and the results show that the HVCMM model outperforms other baseline models on the ROUGE metric. With the help of the HVCMM model, teachers can improve the efficiency of reflection after class and improve the teaching level. Students can review the key content to strengthen their understanding of what they have learned with the help of the class minutes automatically generated by the model.

Enhancing Task-Oriented Dialogue Systems through Synchronous Multi-Party Interaction and Multi-Group Virtual Simulation

This paper presents two innovative approaches: a synchronous multi-party dialogue system that engages in simultaneous interactions with multiple users, and multi-group simulations involving virtual user groups to evaluate the resilience of this system. Unlike most other chatbots that communicate with each user independently, our system facilitates information gathering from multiple users and executes 17 administrative tasks for group requests adeptly by leveraging a state machine-based framework for complete control over dialogue flow and a large language model (LLM) for robust context understanding. Assessing such a unique dialogue system poses challenges, as it requires many groups of users to interact with the system concurrently for an extended duration. To address this, we simulate various virtual groups using an LLM, each comprising 10–30 users who may belong to multiple groups, in order to evaluate the efficacy of our system; each user is assigned a persona and allowed to interact freely without scripts. As a result, our system shows average success rates of 87% for task completion and 89% for natural language understanding. Comparatively, our virtual simulation, which has an average success rate of 80%, is juxtaposed with a group of 15 human users, depicting similar task diversity and error trends. To our knowledge, it is the first work to show the LLM’s potential in both task execution and the simulation of a synchronous dialogue system to fully automate administrative tasks.

“I know this language, that language, and my language”

Abstract Adopting a longitudinal dynamic perspective on the field of Family Language Policy (FLP), combined with insights from language socialization, the current study explores five Iranian immigrant family members’ language ideologies, practices and policies and aims to unfold the processes through which the language ideologies and policies are instantiated and negotiated. Particular foci are the evolving nature of family language ideologies and practices, emphasizing the influence of children’s language ideologies and societal interactions. Through detailed analyses of multiparty family interactions, the study argues the significance of considering children’s language ideologies as integral to understanding of their language practices, moving beyond a simplistic attribution of actions to mere ‘agency’.

Hypothetical active voicing in AVATAR therapy

This paper explores AVATAR therapy, an innovative therapeutic intervention for people experiencing auditory verbal hallucinations (AVH), where participants interact with their ‘voice’ in the form of a co-designed visual representation of their voice. Previous research has found AVATAR therapy to be successful in its aim to initiate change in the participant’s relationship to and experience of their voice, but no study has explored the interactional practices employed as they unfold in this complex multiparty interaction between therapist, participant and avatar voiced by the therapist. We build on previous research and explore what it is about this interactional technique that might be contributing to the efficacy of this therapy. Using conversation analysis, we examine the therapist’s use of ‘hypothetical active voicing’ (HAV), where the therapist formulates a proposed turn at talk in their interaction with the participant for the participant to reformulate in their next turn with the avatar. We show how, unlike in traditional dyadic therapeutic settings, participants are immediately able to enact the therapist-suggested dialogue. We find that as the therapist increases their assertiveness, control and precision in the delivery of HAV, so does the participant increase their assertiveness towards the avatar, thus contributing to the aims of the therapy

Research on value creation path of logistics platform under the background of digital ecosystem: Based on SEM and fsQCA methods

Digitalization and platform ecosystem have subverted the value creation mode between traditional logistics actors and brought new value additions. Few studies have been done to investigate the influencing factors prompting such value creation of digital platform ecological cooperation in the Chinese logistics industry. The study bridges the gap by using structural equation modeling and fuzzy-set qualitative comparative analysis methods to investigate data from 347 Chinese logistics firms cooperating under the logistics platform. Findings:(1) identification of customer value proposition, complementary cooperation, and multi-party interaction had an indirect influence on logistics service value realization. While transaction cost reduction and digital innovation had a direct influence. (2) Digital capabilities have a positive moderating effect on the relationship between transaction cost reduction and value realization. (3) Comparing the configurations of value realization and ～value realization, it is found that enterprises cannot create value if they only save costs without operation, and should improve digital capabilities to reduce transaction costs. At the same time, the identification of customer value proposition is the source of value creation.

Competitive points in Mandarin-speaking multiparty interaction: Speakership and epistemics

This study explores the use of hand points in competitive epistemic and speakership environments in Mandarin-speaking conversation. It examines incoming speakers’ points at a current speaker. This study employs multimodal conversation-analytic methods to analyse 334 min of triadic interaction. 40 points directed at a current speaker by an incoming speaker are the focus of analysis. This study finds that these points indicate an upcoming bid for the floor, thereby carrying out self-selection for speakership. In addition, the turn introduced by the incoming speaker's point suggests that they know at least as much as the current speaker about the issue at hand, and can implicate epistemic and speakership competition. As such, these points foreshadow delivery of a weakly aligning turn or a disaligning turn, and may accomplish either affiliation or disaffiliation. This study generates new knowledge about the interactional functions of points in multiparty conversation, and suggests potential phenomena for cross-linguistic comparison.

Third-Generation Heritage Spanish Acquisition and Socialization: Word Learning and Overheard Input in an L.A.-Based Mexican Family

This case study examines overheard speech in a third-generation heritage Spanish Mexican family. It presents Spanish use longitudinally and describes overheard Spanish word use in interaction. Transcribed on CLAN to create a plurilingual corpus, ethnographic video data consisted of 24 h across three sampling periods, yielding nearly 30,000 Spanish, English, and language mixed utterances. Quantitative analyses indicate strong Spanish use in the first sample, before dropping. Qualitative descriptions show the third-generation target-child’s attunement to overheard Spanish, and her agency to use Spanish. Overheard input helps her use Spanish words, influencing her social encounters. This paper examines what we coded as overheard input in heritage language acquisition and socialization research. The language practices of one multigenerational Mexican family in California are explored, accounting for how their language practices in multiparty interaction co-create meaning, and how they help a third-generation child use Spanish words grounded in daily experiences. The findings contribute to the discussion of bilingualism in general and definitions of heritage bilingualism in particular. The results underscore the understudied role of overhead speech produced by a diversity of multigenerational family members and word learning. Participation frameworks are dynamically constructed by all participants as permeable, inclusive, and engage the children’s use of inherited bilingual and bicultural practices, suggesting that heritage bilingualism is not just about abstract grammar.

When the Mess Is the Message: Simultaneous Signing in an ASL Multiparty Interaction

Abstract: This article presents the results of a microanalysis of sequences when two or more deaf signers overlap. This contribution focuses on cooperative overlap —which consists of extended moments of turn-exchanges that move along the unfolding discourse while cohering interlocutors in relation to each other. This aspect of signed interaction has received little scholarly attention but appears to be one crucial part of the "enchronic glue" (Enfield 2022) of language binding interactions (and interactants) together. Little is known about the specific underpinnings of these seemingly messy interactive moments. We seek here to understand how such extended overlaps are achieved by deaf coparticipants where the outcome results in a forward move of the interaction. A total of thirty-three instances of collaborative overlapping signing were identified, accounting for approximately seven out of the seventy-five minute multiparty interaction between four American Sign Language (ASL) signers. Out of these, specific interactional sequences were analyzed using techniques from ethnomethodology and conversation analysis to answer the following question: What embodied practices are used to negotiate overlaps while maintaining interactional synchrony? In addition to lexical repetition strategies identified in earlier work, these closely timed sequences of overlap are successful because they engage multiple articulators coordinated across signers. Further, it appears that these practices that are symbolic of the joint coordination of co-constructed discourse are a behavior that signers both willingly engage in to achieve specific interactive tasks, namely clarification-seeking, confirmation, and coordinated depiction. Results do not only account for how deaf signers achieve collaborative overlaps but also reveal the interactive reasons why signers engage in these seemingly messy moments of interaction.

Co-constructing parenthood in multiparty interaction: Orienting to parents’ rights and responsibilities to act on behalf of others

Drawing on naturally occurring, multiparty interactional data involving parents, children, and third parties (e.g. friends and relatives), this conversation analytic study investigates how the status of ‘parent’ is co-constructed on a moment-by-moment basis in the course of everyday interaction. The analysis focuses on participant orientation to parents’ rights to act on behalf of their children and third parties, through which parental entitlements, responsibilities, and authority are invoked. Specifically, interaction participants orient to parents as having primary rights to know about their children, determine their courses of action, and take primary responsibilities for their behaviors. Parents also confirm and ratify these category ascriptions by acting on behalf of their children and third parties, demonstrating with actions that they are capable of carrying out their rights and responsibilities as parents. The findings shed light on how the practical relevance of the ‘parent’ membership shapes the sequential unfolding of multiparty interaction.

Dialogue acts enhanced extract–abstract framework for meeting summarization

Meeting summarization is challenging due to complex multi-party interactions with spontaneous utterances. This paper tries to mitigate this challenge by leveraging dialogue acts (DAs). We propose Dialogue Acts enhanced extract–abstract framework for meeting Summarization (DASum). We jointly train meeting summarization with DAs classification to enable our model get aware of DAs and better understand dialogue interactions. Moreover, our DASum model uses DAs to measure the salience of an utterance in the extractor and as prefix prompt to guide how to integrate extracted utterances in the abstractor. We conducted experiments on two public datasets, ICSI (for meeting summarization) and QMSum (for query-based meeting summarization), which compress the meeting transcripts (average 9000 tokens) into a general summary (no more than 700 tokens) or a specific summary (no more than 150 tokens) for a query. Experimental results show our DASum achieves promising results on the ICSI (46.41/ 11.52/ 43.82/ 5.86) and QMSum (36.37/ 11.71/ 32.07/ 19.18) datasets in terms of ROUGE-1/ ROUGE-2/ ROUGE-L/ BERTScore.

Analysis of Gaze, Head Orientation, and Joint Attention in Autism With Triadic VR Interviews.

Effective use of gaze and head orientation can strengthen the sense of inclusion in multi-party interactions, including job interviews. Not making significant eye contact with the interlocutors, or not turning towards them, may be interpreted as disinterest, which could worsen job interview outcomes. This study aims to support the situational solo practice of gaze behavior and head orientation using a triadic (three-way) virtual reality (VR) job interview simulation. The system lets users encounter common interview questions and see how they share attention among the interviewers based on their conversational role (speaking or listening). Given the yaw and position readings of the VR headset, we use a machine learning-based approach to analyze head orientations relative to the interviewers in the virtual environment, and achieve low angular error in a low complexity way. We examine the degree to which interviewer backchannels trigger attention shifts or behavioral mirroring and investigate the social modulation of gaze and head orientation for autistic and non-autistic individuals. In both speaking and listening roles, the autistic participants gazed at, and oriented towards the two virtual interviewers less often, and they displayed less behavioral mirroring (mirroring the head turn of one avatar towards another) compared to the non-autistic participants.

Using conversation analysis to identify unresponsiveness in peer interactions in inclusive groups

Background: Previous research regarding unresponsiveness in peer interaction, including participants on the autism spectrum (AS), is mainly based on predefined categorizations of unresponsiveness; thus, there is a need for conversation analytic research to examine unresponsiveness from participants’ perspectives. Method: Multimodal conversation analysis (CA) was applied to examine unresponsiveness in task-focused multiparty peer interactions of an inclusive group, including one participant on the AS. Results: The results showed that it was not meaningful to analyze unresponsiveness in situations where there was no (aligning) response and all participants’ orientations revealed that a response was (not) needed. Instead, participants’ discrepant orientations to the response relevance made unresponsiveness a meaningful issue for participants to negotiate. Discussion/conclusion: The CA approach can be useful for examining unresponsiveness accurately. The combination of both the speaker and recipient(s) orientations to response relevance can be used as a conceptual tool to identify unresponsiveness when it is relevant for the participants.

Privacy-preserving DeepFake face image detection

All the existing models for DeepFake detection focus on plaintext faces. However, outsourced computing is usually considered in practical applications for DeepFake detection and the input data may contain private and sensitive information. Thus, a privacy-preserving model named Secure DeepFake Detection Network (SecDFDNet) is proposed for the first time in this paper. The SecDFDNet uses the additive secret sharing method for secure DeepFake face detection. Specifically, firstly, some multi-party secure interaction protocols are designed for non-linear activation functions, i.e., SecReLU for ReLU function, SecSigm for sigmoid function, SecSpatial for spatial attention, and SecChannel for channel attention. Their security is proved in theory. Our protocols have low communication and space complexity. Then, the SecDFDNet model is proposed by using the designed secure protocols and trained plaintext DeepFake detection network (DFDNet). The experimental results show that the proposed SecDFDNet can detect DeepFake faces without revealing anything of private input, achieve the same accuracies as the plaintext DFDNet and outperform some existing models. The source code is available at https://github.com/imagecbj/Privacy-Preserving-DeepFake-Face-Image-Detection.

"I See What You Mean"-A Case Study of the Interactional Foundation of Building a Working Alliance in Care Decisions Involving an Older Couple Living with Cognitive Decline.

Social workers have a key role in needs assessment meetings with families dealing with dementia, providing information, support, and advocacy, while also assessing needs and making decisions about care services for several parties. These contacts are especially important during the introduction of home care services, where often the person has previously relied on informal support from relatives. The needs assessment process entails the involvement of all present parties, with the aim to reach a mutual agreement, a working alliance, regarding which services to apply for. The aim of this case study is to explore how the participants, by means of different conversational practices, jointly create a working alliance between the different parties in one family. The study provides insights into the process of co-constructing a working alliance in the needs assessment process for elder care services. This article addresses the process by which social workers build a working alliance in a multi-party conversation with a family living with cognitive decline; a meeting that lasted 50 min. In this case study, we benefit from an inductive and detailed conversation analytic methodology. The theoretical framework of working alliances in institutional interaction has informed the analysis. The findings illustrate how the social worker in this case study involves all parties in the decision regarding care services and explores the use of the conversational practices of mitigations, positive framing, adding information, and positioning, as a "we" achieve mutual agreement toward the end of several sequences. Drawing on the results of this case study, we argue that multi-party interaction involving relatives enables diversity in role-taking, where the professional, for instance, can pursue a more empathic role. Also, our results indicate that minimal agreement to a proposal is sufficient in a multi-party interaction involving clients with cognitive decline.

A Reuse-Oriented Visual Smart Contract Code Generator for Efficient Development of Complex Multi-Party Interaction Scenarios

Multi-party interaction scenarios usually involve interaction and collaboration between multiple participants, such as business negotiation, teamwork, etc. In multi-party interaction scenarios, trust mechanisms are needed to ensure the use of rights and the fulfillment of obligations. The use of blockchain and smart contracts to establish a decentralized trust mechanism is a solution. The development process of smart contracts may require experts in different fields to design them together, and it is difficult for non-experts to understand smart contract code. The traditional method of smart contract development is for the developer to manually write the smart contract code, which may lead to low development efficiency and the inability of the scene participants to understand the smart contract code. To address this challenge, we propose a reuse-oriented visual smart contract generator to efficiently develop complex multi-party interaction scenarios. This paper proposes a model based on BPMN that uses smart contracts to express the process of multi-party interaction so that the parties in various domains understand the business requirements. Then, a multi-layer reusable library is proposed and used to generate a visual solidity code. We also integrate code compilation, security checking, and deployment into the target blockchain environment for testing. The function of the tool is verified in the case of flight delay insurance, which shows that it can aid the automated development of smart contracts for multi-party interaction scenarios. At the same time, the feasibility of the tool is verified by five cases selected from the literature, which shows that it can aid the development of smart contracts to achieve a high degree of reusability.

Face Maintenance Rituals and Communication Strategies Used in Puja Astawa’s YouTube Videos

YouTube, a video sharing platform, is famously used these days, not only due to its function as part of entertainment, but also a media of interaction. Unlike, face-to-face interaction or other computer-mediated interaction, YouTube served multi-party interaction and three levels of communication. Its uniqueness leads to questions on whether people engaged in this interaction manage maintaining their interlocutor’s face and what communication strategies are used amid this multi-party interaction considering it may involve people from different social and cultural background. This qualitative research was aimed to analyse the first level communication in YouTube videos by using interactional sociolinguistics approach in order to figure out the types of face maintenance rituals as well as the communication strategies used by the participants in YouTube videos. The data source of the study was Puja Astawa’s YouTube videos, one of the most popular content creators in Bali. In collecting the data, documentation method and note taking technique were used. The data were further analysed using the face maintenance theory proposed by Goffman and communication strategies proposed by Canale and Swain. The result of the analysis shows that presentational rituals is the main rituals used by the participants in Puja Astawa’s YouTube videos. In terms of communication strategies, there are three strategies found, namely (1) code switching, (2) repetition, (3) the use of Balinese particles. Code switching is the mostly used strategy while the use of Balinese particles is the least used strategy.