Animated Conversational Agents Research Articles

Animated conversational agents and e-consumer productivity: The roles of agents and individual characteristics

This research presents two studies, in a lab (eye-tracking) and in a natural context, that highlight the effects of interacting with an animated conversational agents (ACA) on the objective and perceived econsumer productivity. Results from Study 1 specify that only objective e-consumer productivity depends on interaction with the ACA. Going further, Study 2 then reveals that individual characteristics affect perceived productivity, either independently from ACA use (for involvement or product familiarity) or in interaction with using the ACA (for Internet skills and need for interaction). These findings highlight the need to personalize websites that display an agent fitting user profiles.

Automated indexing of Internet stories for health behavior change: weight loss attitude pilot study.

BackgroundAutomated health behavior change interventions show promise, but suffer from high attrition and disuse. The Internet abounds with thousands of personal narrative accounts of health behavior change that could not only provide useful information and motivation for others who are also trying to change, but an endless source of novel, entertaining stories that may keep participants more engaged than messages authored by interventionists.ObjectiveGiven a collection of relevant personal health behavior change stories gathered from the Internet, the aim of this study was to develop and evaluate an automated indexing algorithm that could select the best possible story to provide to a user to have the greatest possible impact on their attitudes toward changing a targeted health behavior, in this case weight loss.MethodsAn indexing algorithm was developed using features informed by theories from behavioral medicine together with text classification and machine learning techniques. The algorithm was trained using a crowdsourced dataset, then evaluated in a 2×2 between-subjects randomized pilot study. One factor compared the effects of participants reading 2 indexed stories vs 2 randomly selected stories, whereas the second factor compared the medium used to tell the stories: text or animated conversational agent. Outcome measures included changes in self-efficacy and decisional balance for weight loss before and after the stories were read.ResultsParticipants were recruited from a crowdsourcing website (N=103; 53.4%, 55/103 female; mean age 35, SD 10.8 years; 65.0%, 67/103 precontemplation; 19.4%, 20/103 contemplation for weight loss). Participants who read indexed stories exhibited a significantly greater increase in self-efficacy for weight loss compared to the control group (F 1,107=5.5, P=.02). There were no significant effects of indexing on change in decisional balance (F 1,97=0.05, P=.83) and no significant effects of medium on change in self-efficacy (F 1,107=0.04, P=.84) or decisional balance (F 1,97=0.78, P=.38).ConclusionsPersonal stories of health behavior change can be harvested from the Internet and used directly and automatically in interventions to affect participant attitudes, such as self-efficacy for changing behavior. Such approaches have the potential to provide highly tailored interventions that maximize engagement and retention with minimal intervention development effort.

Gaze awareness in conversational agents

In face-to-face conversations, speakers are continuously checking whether the listener is engaged in the conversation, and they change their conversational strategy if the listener is not fully engaged. With the goal of building a conversational agent that can adaptively control conversations, in this study we analyze listener gaze behaviors and develop a method for estimating whether a listener is engaged in the conversation on the basis of these behaviors. First, we conduct a Wizard-of-Oz study to collect information on a user's gaze behaviors. We then investigate how conversational disengagement, as annotated by human judges, correlates with gaze transition, mutual gaze (eye contact) occurrence, gaze duration, and eye movement distance. On the basis of the results of these analyses, we identify useful information for estimating a user's disengagement and establish an engagement estimation method using a decision tree technique. The results of these analyses show that a model using the features of gaze transition, mutual gaze occurrence, gaze duration, and eye movement distance provides the best performance and can estimate the user's conversational engagement accurately. The estimation model is then implemented as a real-time disengagement judgment mechanism and incorporated into a multimodal dialog manager in an animated conversational agent. This agent is designed to estimate the user's conversational engagement and generate probing questions when the user is distracted from the conversation. Finally, we evaluate the engagement-sensitive agent and find that asking probing questions at the proper times has the expected effects on the user's verbal/nonverbal behaviors during communication with the agent. We also find that our agent system improves the user's impression of the agent in terms of its engagement awareness, behavior appropriateness, conversation smoothness, favorability, and intelligence.

ACA as E-Atmospherics: Is There an “Animation Predisposition” Effect?

Academic literature recognizes animated conversational agents (ACA) as part of e-atmospheric and indicates that the use of such agents on commercial websites might be a source of social presence and experiential value. Within this framework and referring to Serenko’s (2007) work on the degree of animation predisposition, the aim this paper is to study the moderating effect of the degree of animation predisposition on the relationship between social interactivity and experiential value. We, we propose a six hypotheses research model that we test by using SmartPLS2. The results confirm the importance of e-atmospherics in the construction of a pleasant experience on commercial websites and indicate a moderating effect of degree of animation predisposition.

Scripting Animated Conversational Agents in an Internet Multi-Agent Advertising System

This study is aimed to develop an agent-based Internet advertising system. This system allows consumers actively announce requirements. Therefore, advertisers are easier to find and analyze potential consumers for designing appropriate advertisements. Furthermore, the designed Internet advertising system supports a XML-based interactive drama markup language (IDML) for advertisers to edit animated conversational agents in advertisements for expressively interacting with consumers. IDML proposed in our previous study integrates the concept of interactive drama to support branching narrations that enables the audience to affect an animated agent’s behaviors and also enables an animated agent to present a variety of dialogues. We also developed the advertising system attached to an Internet e-novel community. The empirical statistics collected from the e-novel community shows that this example system is helpful for novelists to design interactive advertisements to attract potential readers.

Speech and face-to-face communication – An introduction

This issue focuses on face-to-face speech communication. Research works have demonstrated that this communicative situation is essential to language acquisition and development (e.g. naming). Face-to-face communication is in fact much more than speaking and speech is greatly influenced both in substance and content by this essential form of communication. Face-to-face communication is multimodal: interacting involves multimodality and nonverbal communication to a large extent. Speakers not only hear but also see each other producing sounds as well as facial and more generally body gestures. Gaze together with speech contribute to maintain mutual attention and to regulate turn-taking for example. Moreover, speech communication involves not only linguistic but also psychological, affective and social aspects of interaction. Face-to-face communication is situated: the true challenge of spoken communication is to take into account and integrate information not only from the speakers but also from the entire physical environment in which the interaction takes place. The communicative setting, the “task” in which the interlocutors are involved, their respective roles and the environmental conditions of the conversation indeed greatly influence how the spoken interaction unfolds. The present issue aims at synthesizing the most recent developments in this topic considering its various aspects from complementary perspectives: cognitive and neurocognitive (multisensory and perceptuo-motor interactions), linguistic (dialogic face to face interactions), paralinguistic (emotions and affects, turn-taking, mutual attention), computational (animated conversational agents, multimodal interacting communication systems).

Embodied conversational agents as conversational partners

AbstractConversational agents are becoming more widespread in computer technologies but there has been little research in how humans interact with them. Two eye tracking studies investigated how humans distribute eye gaze towards conversational agents in complex tutoring systems. In Study 1, participants interacted with the single‐agent tutoring system AutoTutor. Fixation times showed that the agent received most attention throughout the interaction, even when display size was statistically controlled. In Study 2, participants interacted with iSTART. Fixations were on the relevant agents when these agents spoke. Both studies provided evidence that humans regard animated conversational agents as conversational partners in the communication process. Copyright © 2008 John Wiley & Sons, Ltd.

AutoTutor: An Intelligent Tutoring System With Mixed-Initiative Dialogue

AutoTutor simulates a human tutor by holding a conversation with the learner in natural language. The dialogue is augmented by an animated conversational agent and three-dimensional (3-D) interactive simulations in order to enhance the learner's engagement and the depth of the learning. Grounded in constructivist learning theories and tutoring research, AutoTutor achieves learning gains of approximately 0.8 sigma (nearly one letter grade), depending on the learning measure and comparison condition. The computational architecture of the system uses the .NET framework and has simplified deployment for classroom trials.

Social cues in animated conversational agents

In human-computer interaction people often interpret the interaction with the computer as interactions with humans. The social agency theory suggests that social cues like the face and voice of the agent motivate this interpretation. In two off-line experiments in which comprehension scores and liking ratings were collected, we found that participants preferred natural agents with natural voices, as predicted by the social-cue hypothesis. Although female agents with male voices formed an exception, this was explained by a stereotype effect. These findings support the social-cue hypothesis and the social agency theory that human characteristics are applied in the perception of computational animated conversational agents. Copyright © 2005 John Wiley & Sons, Ltd.

Human use regulatory affairs advisor (HURAA): learning about research ethics with intelligent learning modules.

The Human Use Regulatory Affairs Advisor (HURAA) is a Web-based facility that provides help and training on the ethical use of human subjects in research, based on documents and regulations in United States federal agencies. HURAA has a number of standard features of conventional Web facilities and computer-based training, such as hypertext, multimedia, help modules, glossaries, archives, links to other sites, and page-turning didactic instruction. HURAA also has these intelligent features: (1) an animated conversational agent that serves as a navigational guide for the Web facility, (2) lessons with case-based and explanation-based reasoning, (3) document retrieval through natural language queries, and (4) a context-sensitive Frequently Asked Questions segment, called Point & Query. This article describes the functional learning components of HURAA, specifies its computational architecture, and summarizes empirical tests of the facility on learners.

Specifying and animating facial signals for discourse in embodied conversational agents

AbstractPeople highlight the intended interpretation of their utterances within a larger discourse by a diverse set of non‐verbal signals. These signals represent a key challenge for animated conversational agents because they are pervasive, variable, and need to be coordinated judiciously in an effective contribution to conversation. In this paper, we describe a freely available cross‐platform real‐time facial animation system, RUTH, that animates such high‐level signals in synchrony with speech and lip movements. RUTH adopts an open, layered architecture in which fine‐grained features of the animation can be derived by rule from inferred linguistic structure, allowing us to use RUTH, in conjunction with annotation of observed discourse, to investigate the meaningful high‐level elements of conversational facial movement for American English speakers. Copyright © 2004 John Wiley & Sons, Ltd.

Let's talk! Socially intelligent agents for language conversation training

This paper promotes socially intelligent animated agents for the pedagogical task of English conversation training for native speakers of Japanese. As a novel feature, role awareness is introduced to animated conversational agents, that are by non-strong affective reasoners, but otherwise often lack the competence observed in humans. In particular, humans may easily adjust their behavior depending on their respective role in a setting, whereas their synthetic pendants tend to be driven mostly by emotions and personality. Our main contribution is the incorporation of a social filter to mental models of animated agents. This program may qualify an agent's expression of its emotional state by the contest, thereby enhancing the agent's believability as a conversational partner. Our implemented system is web-based and demonstrates socially aware animated agents in a virtual coffee shop environment. An experiment with our conversation system shows that users consider socially aware agents as more natural than agents that violate conventional practices.

The power of a nod and a glance: Envelope vs. emotional feedback in animated conversational agents

In this article we describe results froman experiment of user interaction with autonomous , human - like ( humanoid ) conversational agents . We hypothesize that for embodied conversational agents , nonverbal behaviors related to the process of conversation , what we call envelope feedback, is much more important than other feedback , such as emotional expression . We test this hypothesis by having subjects interact with three autonomous agents , all capable of full - duplex multimodal interaction: able to generate and recognize speech , intonation , facial displays , and gesture . Each agent , however , gave a different kind of feedback: ( 1 ) content - related only , ( 2 ) content + envelope feedback , and ( 3 ) content + emotional . Content-related feedback includes answering questions and executing commands; envelope feedback includes behaviors such as gaze , manual beat gesture , and head movements; emotional feedback includes smiles and looks of puzzlement . Subjects' evaluations of the systemwere collected with a questionnaire , and videotapes of their speech patterns and behaviors were scored according to how often the users repeated themselves , how often they hesitated , and how often they got frustrated . The results confirmour hypothesis that envelope feedback is more important in interaction than emotional feedback and that envelope feedback plays a crucial role in supporting the process of dialog . A secondary result fromthis study shows that users give our multimodal conversational humanoids very high ratings of lifelikeness and fluidity of interaction when the agents are capable of giving such feedback .