Virtual Agents Research Articles

Effects of virtual agent interactivity on pro-environmental behavior promotion

To promote pro-environmental behavior (PEB), this study specifically assessed the feeling of a desire to make oneself look better, or to be concerned about one's own “reputation” when observed by an interactive virtual agent. For this study, a controlled experiment was conducted with an interactive agent and a non-interactive agent as experiment conditions to test whether interaction with an agent promotes concern about reputation and PEB. Specifically, this study was conducted to evaluate the following two hypotheses. (Hypothesis 1): A person tends to be more concerned about their own personal reputation when contacting an interactive agent as an observer than when contacting a non-interactive agent. (Hypothesis 2): A person is more likely to perform PEB when contacting an interactive agent than when contacting a non-interactive agent. Before contacting agents, experiment participants were assessed to ascertain their degree of PEB. After interacting with either agent for 5 min, the degree of PEB performed was measured again. The improvement was then evaluated. Then, a questionnaire was administered to assess reputational concern. The questionnaire results supported Hypotheses 1 and 2 (p < .01) and clarified that agent interaction induces concern for reputation in the judgment of an agent: even a non-human agent. Results also demonstrated the importance of interaction for PEB promotion.

Deep Person Generation: A Survey from the Perspective of Face, Pose, and Cloth Synthesis

Deep person generation has attracted extensive research attention due to its wide applications in virtual agents, video conferencing, online shopping, and art/movie production. With the advancement of deep learning, visual appearances (face, pose, cloth) of a person image can be easily generated on demand. In this survey, we first summarize the scope of person generation, and then systematically review recent progress and technical trends in identity-preserving deep person generation, covering three major tasks: talking-head generation (face), pose-guided person generation (pose), and garment-oriented person generation (cloth). More than two hundred papers are covered for a thorough overview, and the milestone works are highlighted to witness the major technical breakthrough. Based on these fundamental tasks, many applications are investigated, e.g., virtual fitting, digital human, and generative data augmentation. We hope this survey could shed some light on the future prospects of identity-preserving deep person generation, and provide a helpful foundation for full applications towards the digital human.

Older adults' experience with virtual conversational agents for health data collection.

Virtual conversational agents (i.e., chatbots) are an intuitive form of data collection. Understanding older adults' experiences with chatbots could help identify their usability needs. This quality improvement study evaluated older adults' experiences with a chatbot for health data collection. A secondary goal was to understand how perceptions differed based on length of chatbot forms. After a demographic survey, participants (≥60 years) completed either a short (21 questions), moderate (30 questions), or long (66 questions) chatbot form. Perceived ease-of-use, usefulness, usability, likelihood to recommend, and cognitive load were measured post-test. Qualitative and quantitative analyses were used. A total of 260 participants reported on usability and satisfaction metrics including perceived ease-of-use (5.8/7), usefulness (4.7/7), usability (5.4/7), and likelihood to recommend (Net Promoter Score = 0). Cognitive load (12.3/100) was low. There was a statistically significant difference in perceived usefulness between groups, with a significantly higher mean perceived usefulness for Group 1 than Group 3. No other group differences were observed. The chatbot was perceived as quick, easy, and pleasant with concerns about technical issues, privacy, and security. Participants provided suggestions to enhance progress tracking, edit responses, improve readability, and have options to ask questions. Older adults found the chatbot to be easy, useful, and usable. The chatbot required low cognitive load demonstrating it could be an enjoyable health data collection tool for older adults. These results will inform the development of a health data collection chatbot technology.

FaceMe: An agent-based social game using augmented reality for the emotional development of children with autism spectrum disorder

Children with autism spectrum disorder (ASD) experience impaired emotional development, which severely affects their social communication. Although some assistive technologies are effective in helping children with ASD to improve their social skills, they are generally considered to be unable to engage children in real-world social situations. This paper presents “FaceMe,” which is a virtual agent social game based on an augmented reality (AR) technology that supports children's emotional development. FaceMe uses a virtual agent, a set of tangible toolkits, and multilevel game mechanics to teach children about basic facial expressions in social scenarios and improve their emotional and communication skills. Empirical results show that the FaceMe virtual agent evoked active social behaviors in children with ASD who subsequently improved their ability to understand facial expressions. The FaceMe design guidelines for creating the virtual agent in the context of AR are presented, which opens up an interesting domain for future games using AR to support children's learning.

Do you like me? Behavioral and physical features for socially and emotionally engaging interactive systems

With the aim to give an overview of the most recent discoveries in the field of socially engaging interactive systems, the present paper discusses features affecting users' acceptance of virtual agents, robots, and chatbots. In addition, questionnaires exploited in several investigations to assess the acceptance of virtual agents, robots, and chatbots (voice only) are discussed and reported in the Supplementary material to make them available to the scientific community. These questionnaires were developed by the authors as a scientific contribution to the H2020 project EMPATHIC (http://www.empathic-project.eu/), Menhir (https://menhir-project.eu/), and the Italian-funded projects SIROBOTICS (https://www.exprivia.it/it-tile-6009-si-robotics/) and ANDROIDS (https://www.psicologia.unicampania.it/android-project) to guide the design and implementation of the promised assistive interactive dialog systems. They aimed to quantitatively evaluate Virtual Agents Acceptance (VAAQ), Robot Acceptance (RAQ), and Synthetic Virtual Agent Voice Acceptance (VAVAQ).

Automatic landmark identification in cone-beam computed tomography.

To present and validate an open-source fully automated landmark placement (ALICBCT) tool for cone-beam computed tomography scans. One hundred and forty-three large and medium field of view cone-beam computed tomography (CBCT) were used to train and test a novel approach, called ALICBCT that reformulates landmark detection as a classification problem through a virtual agent placed inside volumetric images. The landmark agents were trained to navigate in a multi-scale volumetric space to reach the estimated landmark position. The agent movements decision relies on a combination of DenseNet feature network and fully connected layers. For each CBCT, 32 ground truth landmark positions were identified by 2 clinician experts. After validation of the 32 landmarks, new models were trained to identify a total of 119 landmarks that are commonly used in clinical studies for the quantification of changes in bone morphology and tooth position. Our method achieved a high accuracy with an average of 1.54 ± 0.87 mm error for the 32 landmark positions with rare failures, taking an average of 4.2second computation time to identify each landmark in one large 3D-CBCT scan using a conventional GPU. The ALICBCT algorithm is a robust automatic identification tool that has been deployed for clinical and research use as an extension in the 3D Slicer platform allowing continuous updates for increased precision.

Synthesizing Game Levels for Collaborative Gameplay in a Shared Virtual Environment

We developed a method to synthesize game levels that accounts for the degree of collaboration required by two players to finish a given game level. We first asked a game level designer to create playable game level chunks. Then, two artificial intelligence (AI) virtual agents driven by behavior trees played each game level chunk. We recorded the degree of collaboration required to accomplish each game level chunk by the AI virtual agents and used it to characterize each game level chunk. To synthesize a game level, we assigned to the total cost function cost terms that encode both the degree of collaboration and game level design decisions. Then, we used a Markov-chain Monte Carlo optimization method, called simulated annealing, to solve the total cost function and proposed a design for a game level. We synthesized three game levels (low, medium, and high degrees of collaboration game levels) to evaluate our implementation. We then recruited groups of participants to play the game levels to explore whether they would experience a certain degree of collaboration and validate whether the AI virtual agents provided sufficient data that described the collaborative behavior of players in each game level chunk. By collecting both in-game objective measurements and self-reported subjective ratings, we found that the three game levels indeed impacted the collaboration gameplay behavior of our participants. Moreover, by analyzing our collected data, we found moderate and strong correlations between the participants and the AI virtual agents. These results show that game developers can consider AI virtual agents as an alternative method for evaluating the degree of collaboration required to finish a game level.

The promise and peril of interactive embodied agents for studying non-verbal communication: a machine learning perspective.

In face-to-face interactions, parties rapidly react and adapt to each other's words, movements and expressions. Any science of face-to-face interaction must develop approaches to hypothesize and rigorously test mechanisms that explain such interdependent behaviour. Yet conventional experimental designs often sacrifice interactivity to establish experimental control. Interactive virtual and robotic agents have been offered as a way to study true interactivity while enforcing a measure of experimental control by allowing participants to interact with realistic but carefully controlled partners. But as researchers increasingly turn to machine learning to add realism to such agents, they may unintentionally distort the very interactivity they seek to illuminate, particularly when investigating the role of non-verbal signals such as emotion or active-listening behaviours. Here I discuss some of the methodological challenges that may arise when machine learning is used to model the behaviour of interaction partners. By articulating and explicitly considering these commitments, researchers can transform 'unintentional distortions' into valuable methodological tools that yield new insights and better contextualize existing experimental findings that rely on learning technology. This article is part of a discussion meeting issue 'Face2face: advancing the science of social interaction'.

Evaluating gaze behaviors as pre-touch reactions for virtual agents.

Reaction behaviors by human-looking agents to nonverbal communication cues significantly affect how they are perceived as well as how they directly affect interactions. Some studies have evaluated such reactions toward several interactions, although few approached before-touch situations and how the agent's reaction is perceived. Specifically, it has not been considered how pre-touch reactions impact the interaction, the influence of gaze behavior in a before-touch situation context and how it can condition the participant's perception and preferences in the interaction. The present study investigated the factors that define pre-touch reactions in a humanoid avatar in a virtual reality environment and how they influence people's perceptions of the avatars. We performed two experiments to assess the differences between approaches from inside and outside the field of view (FoV) and implemented four different gaze behaviors: face-looking, hand-looking, face-then-hand looking and hand-then-face looking behaviors. We also evaluated the participants' preferences based on the perceived human-likeness, naturalness, and likeability. In Experiment 1, we evaluated the number of steps in gaze behavior, the order of the gaze-steps and the gender; Experiment 2 evaluated the number and order of the gaze-steps. A two-step gaze behavior was perceived as more human and more natural from both inside and outside the field of view and that a face-first looking behavior when defining only a one-step gaze movement was preferable to hand-first looking behavior from inside the field of view. Regarding the location from where the approach was performed, our results show that a relatively complex gaze movement, including a face-looking behavior, is fundamental for improving the perceptions of agents in before-touch situations. The inclusion of gaze behavior as part of a possible touch interaction is helpful for developing more responsive avatars and gives another communication channel for increasing the immersion and enhance the experience in Virtual Reality environments, extending the frontiers of haptic interaction and complementing the already studied nonverbal communication cues.

The Impact of the Digital Environment on Younger Students and Adolescents: Modern Trends in the Context of War and Crises

&lt;p&gt;&lt;strong&gt;&lt;em&gt;The aim of the study&lt;/em&gt;&lt;/strong&gt;&lt;em&gt; is to determine the role of parents in controlling the interaction of younger school students (primary school) and adolescents with the digital environment during the duration of the martial law, to examine the emotional, evaluative and personal-semantic relations in the family environment that arise as a result of parental control over child’s digital interaction.&lt;/em&gt;&lt;strong&gt;&lt;em&gt; &lt;/em&gt;&lt;/strong&gt;&lt;em&gt;&lt;/em&gt;&lt;/p&gt;&lt;p&gt;&lt;strong&gt;&lt;em&gt;Research methodology.&lt;/em&gt;&lt;/strong&gt;&lt;em&gt; To solve the research tasks, a number of special research methods were used: unstructured interviews, focus group, questionnaires, and Likert scale.&lt;/em&gt;&lt;/p&gt;&lt;p&gt;&lt;strong&gt;&lt;em&gt;Results.&lt;/em&gt;&lt;/strong&gt;&lt;em&gt; The study of the vector of children's interaction with the digital / family environment during the active military invasion in Ukraine made it possible to state there are different trends in the assessment by children of different ages of the ease/difficulty of communication with parents (primary school children mark their own communication with parents as “very easy” by 10% more, adolescents, on the contrary, show a decrease in the share of those who find it easy to talk to their parents – by 8%). Aware of the dangerous trends of arbitrary and uncontrolled stay of the child in the digital environment, only 9.3% of parents refused to control network activity due to trust in the child. On the other hand, 57.1% of parents among those who answered that they exercise control “constantly” and “often” stressed that they face conflict situations from time to time. Relationships between parents and children that are based on feelings of kindness, love, care, concern, and respect for each other have a positive influence on the development of the child, determined by meaningful emotional and volitional efforts and manifested in real interpersonal situations.&lt;/em&gt;&lt;/p&gt;&lt;p&gt;&lt;strong&gt;&lt;em&gt;Novelty.&lt;/em&gt;&lt;/strong&gt;&lt;em&gt; The study of the factors of favorable / unfavourable influence of the digital / family environment on the emotional and psychological state of children against the background of active hostilities in Ukraine, clarification of the role of parents in controlling the interaction of younger students and adolescents with the digital environment under martial law is carried out for the first time.&lt;/em&gt;&lt;/p&gt;&lt;p&gt;&lt;strong&gt;&lt;em&gt;The practical significance.&lt;/em&gt;&lt;/strong&gt;&lt;em&gt; The obtained results will contribute to the study of the impact of the social environment on the development of children, taking into account different age groups of children; defining a system of indicators for the successful impact of the digital environment on children of different age groups.&lt;/em&gt;&lt;/p&gt;&lt;p&gt;&lt;strong&gt;&lt;em&gt;Key words:&lt;/em&gt;&lt;/strong&gt;&lt;em&gt; digital environment, network activity, social networks, virtual agents of socialization.&lt;/em&gt;&lt;em&gt;&lt;/em&gt;&lt;/p&gt;&lt;div&gt;&lt;div&gt;&lt;p&gt; &lt;/p&gt;&lt;/div&gt;&lt;/div&gt;

Predicting Depression Severity from Spontaneous Speech as Prompted by a Virtual Agent

IntroductionOne of the major challenges in clinical psychiatry remains the absence of well established objective measures of symptoms’ severity. Clinical insights are mainly provided through keen behavioral observation and subjective questionnaires and scales.ObjectivesThe aim of this paper is to predict depression severity through speech using the features extracted from the speech as provided by participants during a semi-structured dialogue with a virtual avatar.MethodsWe use data from a subset of the DAICWOZ dataset consisting in 142 dialogues between participants and a virtual avatar during which the avatar uses several prompts to maintain a conversation with the participant. The avatar uses prompts involving the topics of travel, dream jobs, and memorable experiences. From the speech generated from the dialogue, we extract participant utterances separated by prompt and extract features from the three sets of transcripts. We extract content features from the transcript and acoustic features from the excerpt corresponding to the speech from the participant for the prompt in question.We perform regression experiments on the PHQ8 items using the features extracted from each set of transcripts. Furthermore, we combine the features extracted from each set of transcripts and compute partial spearman correlations between them and the PHQ8 items using gender as a covariate.ResultsWith our best regression model we obtain an R2 of 0.1, explaining 10% of the variance in the PHQ total score. Additionally, we obtain a mean absolute error of 1.25, suggesting that the regressor can detect with more or less precision clinically meaningful differences in depression severity between participants. Partial correlations between the total score and the features show significant correlations between features dependent on the amount of speech generated by each participant, along with the complexity of syntactic structures used.ConclusionsAutomatic analysis of spontaneous speech could help with the detection and monitoring of signs of depression. By combining the use of this technology with timely intervention strategies for instance provided by a virtual agent it could contribute to timely prevention.Disclosure of InterestA. König: None Declared, M. Mina Employee of: ki:elements GmbH, S. Schäfer Employee of: ki:elements GmbH, N. Linz Shareolder of: ki:elements GmbH, Employee of: ki:elements GmbH, J. Tröger Shareolder of: ki:elements GmbH, Employee of: ki:elements GmbH

Flocks, Mobs, and Figure Eights: Swarming as a Lemniscatic Arch

Inspired by the natural mobbing behavior of birds, this work presents a novel, quasi-distributed swarming strategy called the Dynamic Lemniscatic Arch. It resolves the problem of producing globally-stable, evenly-spaced lemniscate (or, figure-eight) trajectories while relying on local interactions only. Such trajectories are advantageous in applications where energy consumption and mechanical strain must be minimized. Previous work in lemniscate curves has typically relied on predetermined trajectories, rather than on the emergent structure of the swarm. Furthermore, we enrich the traditional 2-dimensional lemniscate plane curve structure by forming an arch in the third dimension. This arch provides more consistent coverage in surveillance type tasks and, with minor variations in parameters, can be used to produce mobbing behavior. The technique relies on time-varying quaternion rotations linked to the positions of dynamically induced virtual agents. We provide a mathematical proof of stability, which demonstrates the swarm converges to the desired geometry. Simulations show that the strategy performs well with multiple agents and in numerous different configurations.

How to establish junior high school English: Development of learning system with neuroscience verification using language processing technology

There is a need for better communication as globalisation continues apace and, therefore, learning a foreign language could be considered more important than ever before. Professor Hirokazu Yokokawa is a psycholinguistic expert based at Kobe University and believes that improving one's English language ability is essential to living better in a global society. He is currently working to utilise knowledge communication technology to develop a learning training system in collaboration with engineering and neurology researchers. When it comes to communicating in a foreign language, Yokokawa's approach is to offer the text during the retelling task so that the learner can focus on training in grammar coding processing as the initial conceptualisation sub-task has been reduced, thereby promoting the automation of language processing and improving social skills. Yokokawa and his collaborators have carried out a user survey to determine users' preferences with regards to feedback on their learning and they plan to conduct further investigations to decipher the type of virtual agent behaviour that is preferred by the majority of users and whether this has an impact on training. The researchers' learning system includes interaction with avatars with a view to increase users' subjective assessment of their own speaking abilities and decrease nervousness associated with interacting with an unfamiliar person, ultimately helping users to feel more at ease.

Pengembangan Agen Virtual Berbasis Telegram Bot Pada SIMANTA Menggunakan Metode Similarity-Sentence Measurement

The use of chatbots is now starting to be widely used to support services from information systems, one of which is SIMANTA. SIMANTA, which is managed by the Department of Business Administration, State Polytechnic of Malang, is used as the management of student final assignments and theses. SIMANTA has been used for 3 years starting from 2019 until now. There are obstacles that have not been overcome by SIMANTA, namely the dependence of students on the SIMANTA admin. With the high demand for using SIMANTA in the Commerce Administration Department, a lot of questions arise about registration, title submissions, guidance information, proposal seminar submissions, proposal seminar schedules, final report exams through social media applications that cannot be handled by the administration. , thus causing problems that cause the information conveyed to be less responsive. This research creates a feature in the form of a chatbot virtual agent on SIMANTA using the Telegram platform. Chatbot is created using BotFather with the name @SIMANTA_BOT. Chatbot utilizes the Similarity-Sentences Measurement method to measure the level of similarity of the questions obtained, and provides responses using the pattern-matching method. With the chatbot feature on SIMANTA, it can help improve services to students, teaching staff, and education staff within the Department of Business Administration.

Exploring user experience and performance of a tedious task through human–agent relationship

Positive human–agent relationships can effectively improve human experience and performance in human–machine systems or environments. The characteristics of agents that enhance this relationship have garnered attention in human–agent or human–robot interactions. In this study, based on the rule of the persona effect, we study the effect of an agent’s social cues on human–agent relationships and human performance. We constructed a tedious task in an immersive virtual environment, designing virtual partners with varying levels of human likeness and responsiveness. Human likeness encompassed appearance, sound, and behavior, while responsiveness referred to the way agents responded to humans. Based on the constructed environment, we present two studies to explore the effects of an agent’s human likeness and responsiveness to agents on participants’ performance and perception of human–agent relationships during the task. The results indicate that when participants work with an agent, its responsiveness attracts attention and induces positive feelings. Agents with responsiveness and appropriate social response strategies have a significant positive effect on human–agent relationships. These results shed some light on how to design virtual agents to improve user experience and performance in human–agent interactions.

Development of a Reinforcement Learning Algorithm to Optimize Corticosteroid Therapy in Critically Ill Patients with Sepsis

The optimal indication, dose, and timing of corticosteroids in sepsis is controversial. Here, we used reinforcement learning to derive the optimal steroid policy in septic patients based on data on 3051 ICU admissions from the AmsterdamUMCdb intensive care database. We identified septic patients according to the 2016 consensus definition. An actor-critic RL algorithm using ICU mortality as a reward signal was developed to determine the optimal treatment policy from time-series data on 277 clinical parameters. We performed off-policy evaluation and testing in independent subsets to assess the algorithm's performance. Agreement between the RL agent's policy and the actual documented treatment reached 59%. Our RL agent's treatment policy was more restrictive compared to the actual clinician behavior: our algorithm suggested withholding corticosteroids in 62% of the patient states, versus 52% according to the physicians' policy. The 95% lower bound of the expected reward was higher for the RL agent than clinicians' historical decisions. ICU mortality after concordant action in the testing dataset was lower both when corticosteroids had been withheld and when corticosteroids had been prescribed by the virtual agent. The most relevant variables were vital parameters and laboratory values, such as blood pressure, heart rate, leucocyte count, and glycemia. Individualized use of corticosteroids in sepsis may result in a mortality benefit, but optimal treatment policy may be more restrictive than the routine clinical practice. Whilst external validation is needed, our study motivates a 'precision-medicine' approach to future prospective controlled trials and practice.

I Asked a ChatGPT to Write an Editorial About How We Can Incorporate Chatbots Into Neurosurgical Research and Patient Care….

I Asked a ChatGPT to Write an Editorial About How We Can Incorporate Chatbots Into Neurosurgical Research and Patient Care….

Human-to-Human Interaction Using Virtual Agent Posing as Another Person

Human-to-Human Interaction Using Virtual Agent Posing as Another Person

Interaction with Virtual Humans and Effect of Emotional Expressions: Anger Matters!

Today we are experiencing a hybrid real-virtual society in which the interaction with virtual humans is normal and "quasi-social". Understanding the way we react to the interaction with virtual agents and the impact of emotions on social dynamics in the virtual world is fundamental. Therefore, in this study we investigated the implicit effect of emotional information by adopting a perceptual discrimination task. Specifically, we devised a task that explicitly required perceptual discrimination of a target while involving distance regulation in the presence of happy, neutral, or angry virtual agents. In two Immersive Virtual Reality experiments, participants were instructed to discriminate a target on the virtual agents' t-shirts, and they had to provide the response by stopping the virtual agents (or themselves) at the distance where they could identify the target. Thus, facial expressions were completely irrelevant to the perceptual task. The results showed that the perceptual discrimination implied a longer response time when t-shirts were worn by angry rather than happy or neutral virtual agents. This suggests that angry faces interfered with the explicit perceptual task people had to perform. From a theoretical standpoint, this anger-superiority effect could reflect an ancestral fear/avoidance mechanism that prompts automatic defensive reactions and bypasses other cognitive processes.

Speech emotion recognition with artificial intelligence for contact tracing in the COVID‐19 pandemic

AbstractIf understanding sentiments is already a difficult task in human‐human communication, this becomes extremely challenging when a human‐computer interaction happens, as for instance in chatbot conversations. In this work, a machine learning neural network‐based Speech Emotion Recognition system is presented to perform emotion detection in a chatbot virtual assistant whose task was to perform contact tracing during the COVID‐19 pandemic. The system was tested on a novel dataset of audio samples, provided by the company Blu Pantheon, which developed virtual agents capable of autonomously performing contacts tracing for individuals positive to COVID‐19. The dataset provided was unlabelled for the emotions associated to the conversations. Therefore, the work was structured using a sort of transfer learning strategy. First, the model is trained using the labelled and publicly available Italian‐language dataset EMOVO Corpus. The accuracy achieved in testing phase reached 92%. To the best of their knowledge, thiswork represents the first example in the context of chatbot speech emotion recognition for contact tracing, shedding lights towards the importance of the use of such techniques in virtual assistants and chatbot conversational contexts for psychological human status assessment. The code of this work was publicly released at: https://github.com/fp1acm8/SER.