‘Am I Understood?’: How the Interplay Between Embodiment and Theory of Mind Behavior Affects LLM-based Conversational Agents on Perceived Trust, Anthropomorphism, Presence, Usability, and User Experience

An introduction is presented in which the editor discusses how virtual business environments can accelerate the advancement of research in fraud investigation and mentions two topics within the issue including fraud in virtual world and role of Embodied Conversational Agents in interviews.

Stress has become one of the major reasons for many mental health related issues among students of all age groups, which has resulted in devastating personal losses including suicide. Societal and familial pressure to succeed is high, particularly in developing countries where education is highly valued as a key enabler. As part of stress management during the COVID‐19 pandemic, demand for online intelligent virtual advisors has risen and, consequently, the need for personalised explanation that is culturally sensitive to the user's context is essential to improve the user's understanding of and trust in the recommendations provided by the virtual advisor. This paper presents the mAnaging stRess at University embodied conversational agent (ECA) that has been adapted for Indian university students from an explainable agent that was found to help Western students reduce their stress by providing study tips with explanations based on the student's beliefs and/or goals. We conducted a research study with sixty students which measured the impact of providing three different patterns of tailored explanations (belief‐based, goal‐based, and belief and goal‐based explanation) on the students' intentions to change the recommended behaviours and the relationship built with the ECA. The experimental results indicate that there was stress reduction across all student groups provided with different types of explanations. Further, the students showed trust and a good working alliance with the conversational agent, along with an intention to change behaviour across all types of explanations. However, it was observed that the user context played an important role in behaviour change intention and hence explanations could be tailored further, making them culturally more relevant to Indian students. Practitioner notes What is already known about this topic Embodied conversational agents (ECAs) have been mostly developed, applied and shown to be effective in developed countries. Hence, their design and development are mostly guided by the intended user's needs and preferences. In a Western context, ECAs have been found to be beneficial for reducing study stress in university students. There is a pertinent need for use of low cost, effective technology that can aid academic stress reduction in higher educational institutions in developing countries owing to their high youth populations, lack of adequate mental healthcare facilities and associated social stigma. What this paper adds The adaptation and use of ECAs to reduce study stress in higher education students in a developing country is evaluated. The ECA technology is adapted for an Indian context in terms of its physical appearance, colour, speech dialect and dialog content so that it is culturally more aligned to the target population. The ECA engages in an empathic conversation tailored for the Indian students and their COVID‐19 context providing them with explanation‐backed behaviour recommendations that take their beliefs and goals into account. The ECA provides three types of explanation: belief‐only; goal‐only; and both belief and goal. Results of a study carried out in an Indian university with 61 students, randomly assigned to one of the explanation types, to capture their demographics, study stress statistics, behaviour change intentions and trust/working alliance with the conversational agent. The major findings include stress reduction across all explanation groups, development of a positive relationship between the ECA and the students regardless of its explanation pattern, and changes in behaviour intentions across all types of explanations for all recommended behaviours. However, differences in change intentions for certain behaviours indicate further tailoring of explanations is required based on the user context. Implications for practice and/or policy The ECA technology has shown promise in terms of stress reduction amongst Indian students. Higher Education Institutions in developing countries could utilise low‐cost and widely accessible ECAs to overcome lack of access to human‐based support and reluctance to use available services due to stigmatized attitudes to mental health issues. This technology can be further improved and deployed into a larger number of Indian educational institutions leading to a widespread impact on overall student health and wellbeing. Digital technologies to support mental health have become more prominent during the COVID‐19 pandemic, at least in Western countries. The ECA technology evaluated in our study demonstrates its viability and potential value for use in developing countries, with appropriate tailoring.

Embodied conversational agents (ECAs) are advanced human-like interfaces that engage users in natural face-to-face conversations and interactions. These traits position ECAs as innovative tools for delivering interventions for promoting health-related behavior adoption. This includes motivational interviewing (MI), a therapeutic approach that combines brief interventions with motivational techniques to encourage the adoption of healthier behaviors. This study aims to identify the health issues addressed by ECAs delivering MI interventions, explore the key characteristics of these ECAs (eg, appearance, dialogue mechanism, emotional model), analyze the implementation of MI principles and techniques within ECAs, and examine the evaluation methods and primary outcomes of studies that use ECAs providing MI interventions. We conducted a scoping review following the PRISMA-ScR (Preferred Reporting Items for Systematic Reviews and Meta-Analyses Extension for Scoping Reviews) methodology. Our systematic search covered the PubMed, Scopus, IEEE Xplore, ACM Digital, and PsycINFO databases for papers published between January 2008 and December 2022. We included papers describing ECAs developed for delivering MI interventions targeting health-related behaviors and excluded articles that did not describe ECAs with human appearances and without the necessary evaluation or MI explanation. In a multistage process, 3 independent reviewers performed screening and data extraction, and the collected data were synthesized using a narrative approach. The initial search identified 404 articles, of which 3.5% (n=14) were included in the review. ECAs primarily focused on reducing alcohol use (n=5, 36%), took on female representations (n=9, 64%), and gave limited consideration to user ethnicity (n=9, 64%). Most of them used rules-driven dialogue mechanisms (n=13, 93%), include emotional behavior to convey empathy (n=8, 57%) but without an automatic recognition of user emotions (n=12, 86%). Regarding MI implementation, of 14 studies, 3 (21%) covered all MI principles, 4 (29%) included all processes, and none covered all techniques. Most studies (8/14, 57%) conducted acceptability, usability, and user experience assessments, whereas a smaller proportion (4/14, 29%) used randomized controlled trials to evaluate behavior changes. Overall, the studies reported positive results regarding acceptability, usability, and user experience and showed promising outcomes in changes in attitudes, beliefs, motivation, and behavior. This study revealed significant advancements in the use of ECAs for delivering MI interventions aimed at promoting healthier behaviors over the past 15 years. However, this review emphasizes the need for a more in-depth exploration of ECA characteristics. In addition, there is a need for the enhanced integration of MI principles, processes, and techniques into ECAs. Although acceptability and usability have received considerable attention, there is a compelling argument for placing a stronger emphasis on assessing changes in attitudes, beliefs, motivation, and behavior. Consequently, inclusion of more randomized controlled trials is essential for comprehensive intervention evaluations.

A strong debate has ensued in the computing community about whether Embodied Conversational Agents (ECAs) are beneficial and whether we should pursue this direction in interface design. Proponents cite the naturalness and power of ECAs as strengths, and detractors feel that ECAs disempower, mislead, and confuse users. As this debate rages on, relatively little systematic empirical evaluation on ECAs is actually being performed, and the results from this research have been contradictory or equivocal. We propose a framework for evaluating ECAs that can systematize the research. The framework emphasizes features of the agent, the user, and the task the user is performing. Our goal is to be able to make informed, scientific judgments about the utility of ECAs in user interfaces. If intelligent agents can be built, are there tasks or applications for which an ECA is appropriate? Are there characteristics (in appearance, in personality, etc.) the ECA should have? What types of users will be more productive and happy by interacting with an ECA? Our initial experiment within this framework manipulated the ECA’s appearance (realistic human versus iconic object) and the objectivity of the user’s task (editing a document versus deciding what to pack on a trip). We found that the perception of the ECA was strongly influenced by the task while features of the ECA that we manipulated had littlee ect.

BackgroundReminiscence therapy can improve emotional and cognitive engagement in individuals with dementia, thus restoring their sense of self‐worth, self‐esteem, and quality of life (Macleod et al., 2021). Embodied Conversational Agents (ECAs) offer a novel way to facilitate this therapy by providing consistent, personalised, and engaging interactions (Aylett et al., 2024 a&b). Reminiscence technologies should emerge from co‐design methodologies, which emphasise collaboration with users and caregivers to create theory‐driven and user‐led systems that are intuitive, engaging, and impactful. Here, we report on the co‐creation design approach of AMPER.MethodsThe co‐design process incorporated user‐centred design (UCD) and participatory design (PD) methodologies. Workshops and interviews were conducted with caregivers, clinicians, and other stakeholders to define key design requirements. Prototypes were developed iteratively, integrating feedback to refine visual design, interaction flow, and narrative content. ECAs prototypes were tested for their suitability, using scales to measure the reliability of traits (trustworthiness, friendliness, kindness, knowledgeability, amicability, and non‐stereotypical appearance) and engagement. Natural language processing and multimedia prompts were used to enhance personalisation and contextual relevance.ResultsOut of 17 ECAs, 8 were shortlisted by 6 raters. The resulting female (n = 4) and male (n = 4) ECAs attracted high ratings on trustworthiness (87.6/130), friendliness (87.3/130), kindness (87.9/130), knowledgeability (86.4/130), amicability (89.6/130), and non‐stereotypical (82.9/130) from cognitively unimpaired older adults (n = 26 for female ECA and n = 28 for male ECA, age range 59‐87). Trustworthiness showed correlations (Spearman's rho 0.3−0.4) with the other traits but non‐stereotypical. Intra‐class Correlation Coefficient (ICC) was 0.88 (p < .001, 95% CI=0.73‐0.97), suggesting high inter‐rater agreement. Participants reported increased emotional engagement, citing the value of personalised ECAs, narratives, and multimedia integration.ConclusionsThe AMPER co‐creation design approach proved effective in creating tailored ECAs for reminiscence therapy (Aylett et al., 2024b). We provide guidelines for future evaluation of intelligent ECAS in a co‐design context (Aylett et al., 2024a). This research contributes to the growing field of trustworthy autonomous systems for healthcare applications, emphasising interdisciplinarity, user trust, and engagement as critical factors for success.

BackgroundReminiscence therapy can improve emotional and cognitive engagement in individuals with dementia, thus restoring their sense of self‐worth, self‐esteem, and quality of life (Macleod et al., 2021). Embodied Conversational Agents (ECAs) offer a novel way to facilitate this therapy by providing consistent, personalised, and engaging interactions (Aylett et al., 2024 a&b). Reminiscence technologies should emerge from co‐design methodologies, which emphasise collaboration with users and caregivers to create theory‐driven and user‐led systems that are intuitive, engaging, and impactful. Here, we report on the co‐creation design approach of AMPER.MethodsThe co‐design process incorporated user‐centred design (UCD) and participatory design (PD) methodologies. Workshops and interviews were conducted with caregivers, clinicians, and other stakeholders to define key design requirements. Prototypes were developed iteratively, integrating feedback to refine visual design, interaction flow, and narrative content. ECAs prototypes were tested for their suitability, using scales to measure the reliability of traits (trustworthiness, friendliness, kindness, knowledgeability, amicability, and non‐stereotypical appearance) and engagement. Natural language processing and multimedia prompts were used to enhance personalisation and contextual relevance.ResultsOut of 17 ECAs, 8 were shortlisted by 6 raters. The resulting female (n = 4) and male (n = 4) ECAs attracted high ratings on trustworthiness (87.6/130), friendliness (87.3/130), kindness (87.9/130), knowledgeability (86.4/130), amicability (89.6/130), and non‐stereotypical (82.9/130) from cognitively unimpaired older adults (n = 26 for female ECA and n = 28 for male ECA, age range 59‐87). Trustworthiness showed correlations (Spearman's rho 0.3−0.4) with the other traits but non‐stereotypical. Intra‐class Correlation Coefficient (ICC) was 0.88 (p <.001, 95% CI=0.73‐0.97), suggesting high inter‐rater agreement. Participants reported increased emotional engagement, citing the value of personalised ECAs, narratives, and multimedia integration.ConclusionsThe AMPER co‐creation design approach proved effective in creating tailored ECAs for reminiscence therapy (Aylett et al., 2024b). We provide guidelines for future evaluation of intelligent ECAS in a co‐design context (Aylett et al., 2024a). This research contributes to the growing field of trustworthy autonomous systems for healthcare applications, emphasising interdisciplinarity, user trust, and engagement as critical factors for success.

Foreword

A virtual tourist counselor expressing intimacy behaviors: A new perspective to create emotion in visitors and offer them a better user experience?

Traditionally, an optimal embodied conversational agent (ECA) has the same capabilities and appearance as an actual person. This chapter proposes a 'user as assessor' approach to evaluating ECAs that focuses on how ECAs manifest human capabilities independent of actual capabilities that an ECA may possess. Literatures on humans as producers of behavior and humans as interpreters of behavior are leveraged to draw implications for how ECAs should behave to seem most realistic to their human assessors. To illustrate the approach, we answer the question, what will convince a user that an ECA is paying attention to him or her, whether the ECA truly is paying attention or not? 'Apparent attention' is conceptualized in terms of two basic dimensions - selectivity and breadth - and their indicators and impacts. Using the proposed approach, the chapter provides guidelines for how agents, conversational agents, and ECAs can effectively exhibit attention.

Embodied Conversational Agents (ECA) have the potential to bring to life many kinds of information, and in particular textual contents. In this paper, we present a prototype that helps visualizing the relative importance of sentences extracted from medical texts (clinical guidelines aimed at physicians). We propose to map rhetorical structures automatically recognized in the documents to a set of communicative acts controlling the expression of the ECA. As a consequence, the ECA will dramatize a sentence to reflect its perceived importance and degree of recommendation (advice, requirement, open proposal, etc). This prototype is constituted of three sub-systems: i) a text analysis module, ii) an ECA and iii) a mapping module which converts rhetorical structures produced by the text analysis module into nonverbal behaviors driving the ECA animation. This system could help authors of medical texts to reflect on the potential impact of the writing style they have adopted. The use of ECA re-introduces an affective element which won't be captured by other methods for analyzing document style.

There is an ongoing debate on what kind of factors contribute to the general positive user experience while playing a game. The following chapter introduces an experimental setting to measure user experience aroused by facial expression of embodied conversational agents (ECAs). The experimental setup enables to measure the implications of ECAs in three contextual settings called “still,” “animated,” and “interaction.” Within the experiment, artificially generated facial expressions are combined with emotion-eliciting situations and are presented via different presentation platforms. Stimuli (facial expressions/emotion-eliciting situations) are assembled in either consonant (for example, facial expression: “joy,” emotion-eliciting situation: “joy”) or dissonant (for example, facial expression: “joy,” emotion-eliciting situation: “anger”) constellations. The contextual setting called “interaction” is derived from the video games domain, granting an interactive experience of a given emotional situation. The aim of the study is to establish a comparative experimental framework to analyze subjects’ user experience on emotional stimuli in different context dimensions. This comparative experimental framework utilizes theoretical models of emotion theory along with approaches from human–computer interaction to close a gap in the intersection of affective computing and research on facial expressions. Results showed that the interaction situation is rated as providing a better user experience, independent of showing consonant or dissonant contextual descriptions. The “still” setting is given a higher user experience rating than the “animated” setting.

The high prevalence of mental illness is a critical social problem. The limited availability of mental health services is a major factor that exacerbates this problem. One solution is to deliver cognitive behavioral therapy (CBT) using an embodied conversational agent (ECA). ECAs make it possible to provide health care without location or time constraints. One of the techniques used in CBT is Socratic questioning, which guides users to correct negative thoughts. The effectiveness of this approach depends on a therapist's skill to adapt to the user's mood or distress level. However, current ECAs do not possess this skill. Therefore, it is essential to implement this adaptation ability to the ECAs. This study aims to develop and evaluate a method that automatically adapts the number of Socratic questions based on the level of detected psychological distress during a CBT session with an ECA. We hypothesize that this adaptive approach to selecting the number of questions will lower psychological distress, reduce negative emotional states, and produce more substantial cognitive changes compared with a random number of questions. In this study, which envisions health care support in daily life, we recruited participants aged from 18 to 65 years for an experiment that involved 2 different conditions: an ECA that adapts a number of questions based on psychological distress detection or an ECA that only asked a random number of questions. The participants were assigned to 1 of the 2 conditions, experienced a single CBT session with an ECA, and completed questionnaires before and after the session. The participants completed the experiment. There were slight differences in sex, age, and preexperimental psychological distress levels between the 2 conditions. The adapted number of questions condition showed significantly lower psychological distress than the random number of questions condition after the session. We also found a significant difference in the cognitive change when the number of questions was adapted based on the detected distress level, compared with when the number of questions was fewer than what was appropriate for the level of distress detected. The results show that an ECA adapting the number of Socratic questions based on detected distress levels increases the effectiveness of CBT. Participants who received an adaptive number of questions experienced greater reductions in distress than those who received a random number of questions. In addition, the participants showed a greater amount of cognitive change when the number of questions matched the detected distress level. This suggests that adapting the question quantity based on distress level detection can improve the results of CBT delivered by an ECA. These results illustrate the advantages of ECAs, paving the way for mental health care that is more tailored and effective.

This paper addresses the role of conversational memory in Embodied Conversational Agents (ECAs). It describes an investigation into developing such a memory architecture and integrating it into an ECA. ECAs are virtual agents whose purpose is to engage in conversations with human users, typically through natural language speech. While several works in the literature seek to produce viable ECA dialog architectures, only a few authors have addressed the episodic memory architectures in conversational agents and their role in enhancing their intelligence. In this work, we propose, implement, and test a unified episodic memory architecture for ECAs. We describe a process that determines the prevalent contexts in the conversations obtained from the interactions. The process presented demonstrates the use of multiple techniques to extract and store relevant snippets from long conversations, most of whose contents are unremarkable and need not be remembered. The mechanisms used to store, retrieve, and recall episodes from previous conversations are presented and discussed. Finally, we test our episodic memory architecture to assess its effectiveness. The results indicate moderate success in some aspects of the memory-enhanced ECAs, as well as some work still to be done in other aspects.

User interaction with Embodied Conversational Agents (ECA) should involve a significant affective component to achieve realism in communication. This aspect has been studied through different frameworks describing the relationship between user and ECA, for instance alignment, rapport and empathy. We conducted an experiment to explore how an ECA's non-verbal expression can be controlled to respond to a single affective dimension generated by users as input. Our system is based on the mapping of a high-level affective dimension, approach/avoidance, onto a new ECA control mechanism in which Action Units (AU) are activated through a neural network. Since 'approach' has been associated to prefrontal cortex activation, we use a measure of prefrontal cortex left-asymmetry through fNIRS as a single input signal representing the user's attitude towards the ECA. We carried out the experiment with 10 subjects, who have been instructed to express a positive mental attitude towards the ECA. In return, the ECA facial expression would reflect the perceived attitude under a neurofeedback paradigm. Our results suggest that users are able to successfully interact with the ECA and perceive its response as consistent and realistic, both in terms of ECA responsiveness and in terms of relevance of facial expressions. From a system perspective, the empirical calibration of the network supports a progressive recruitment of various AUs, which provides a principled description of the ECA response and its intensity. Our findings suggest that complex ECA facial expressions can be successfully aligned with one high-level affective dimension. Furthermore, this use of a single dimension as input could support experiments in the fine-tuning of AU activation or their personalization to user preferred modalities.

This paper discusses the enactment and evaluation of Embodied Conversational Agents (ECA) capable of affective communication in Collaborative Virtual Environments (CVE) for learning. The CVE discussed is a reconstruction of the European Parliament in Brussels developed using the REVERIE (Real and Virtual Engagement In Realistic Immersive Environment) framework. REVERIE is a framework designed to support the creation of CVEs populated by ECAs capable of natural human-like behaviour, physical interaction and engagement. The ECA provides a tour of the virtual parliament and participates in the learning activity as an intervention mechanism to engage students. The ECA is capable of immediacy behaviour (verbal and non-verbal) and interactions to support a dialogic learning scenario. The design of the ECA is grounded on a theoretical framework that addresses the required characteristics of the ECA to successfully support collaborative learning. In this paper, we discuss the Heuristic Evaluation of the REVERIE ECA which revealed a wealth of usability problems that led to the development of a list of design recommendations to improve their usability, including its immediacy behaviours and interactions. An ECA capable of effectively creating rapport should result in more positive experiences for participants and better learning results for students in dialogic learning scenarios. Future work aims to evaluate this hypothesis in real-world scenarios with teachers and students participating in a shared virtual educational experience.