Embodied Agents Research Articles

Dittos: Personalized, Embodied Agents That Participate in Meetings When You Are Unavailable

Imagine being able to send a personalized embodied agent to meetings you are unable to attend. This paper explores the idea of a Ditto an agent that visually resembles a person, sounds like them, possesses knowledge about them, and can represent them in meetings. This paper reports on results from two empirical investigations: 1) focus group sessions with six groups (n=24) and 2) a Wizard of Oz (WOz) study with 10 groups (n=39) recruited from within a large technology company. Results from the focus group sessions provide insights on what contexts are appropriate for Dittos, and issues around social acceptability and representation risk. The focus group results also provide feedback on visual design characteristics for Dittos. In the WOz study, teams participated in meetings with two different embodied agents: a Ditto and a Delegate (an agent which did not resemble the absent person). Insights from this research demonstrate the impact these embodied agents can have in meetings and highlight that Dittos in particular show promise in evoking feelings of presence and trust, as well as informing decision making. These results also highlight issues related to relationship dynamics such as maintaining social etiquette, managing one's professional reputation, and upholding accountability. Overall, our investigation provides early evidence that Dittos could be beneficial to represent users when they are unable to be present but also outlines many factors that need to be carefully considered to successfully realize this vision.

Designing for social relatedness between stroke survivors and eHealth: ‘Edo’ an embodied coach for stroke rehabilitation in the home context

eHealth solutions at home are gaining interest and relevance in healthcare; however, they face challenges in sustaining motivation for therapy due to difficulties in creating meaningful connections between technology and people receiving care. In this article, we explore how embodied agents in eHealth interventions could be designed to establish a motivating relationship with people in the context of home-based stroke rehabilitation. We studied this potential by referring to the need for social relatedness in Self-Determination-Theory (SDT) and how this translates to the design of embodied agents’ characters, the collaborations they afford and the partnerships they establish over time. In co-creative ideation sessions with stroke survivors, this potential was explored, resulting in an interactive prototype of ‘Edo the Coach’, i.e. an embodied agent for home-based upper extremity stroke rehabilitation that was evaluated by stroke survivors in their home contexts. Based on the insights gained, we reflect on the potential of using SDT for designing embodied agents that foster motivational relationships between patient and agent as an answer to the growing demand for eHealth-supported self-care within an increasingly digitized healthcare context.

How Perception, Actuation, and Communication Impact the Emergence of Collective Intelligence in Simulated Modular Robots.

Modular robots are collections of simple embodied agents, the modules, that interact with each other to achieve complex behaviors. Each module may have a limited capability of perceiving the environment and performing actions; nevertheless, by behaving coordinately, and possibly by sharing information, modules can collectively perform complex actions. In principle, the greater the actuation, perception, and communication abilities of the single module are the more effective is the collection of modules. However, improved abilities also correspond to more complex controllers and, hence, larger search spaces when designing them by means of optimization. In this article, we analyze the impact of perception, actuation, and communication abilities on the possibility of obtaining good controllers for simulated modular robots, that is, controllers that allow the robots to exhibit collective intelligence. We consider the case of modular soft robots, where modules can contract, expand, attach, and detach from each other, and make them face two tasks (locomotion and piling), optimizing their controllers with evolutionary computation. We observe that limited abilities often do not prevent the robots from succeeding in the task, a finding that we explain with (a) the smaller search space corresponding to limited actuation, perception, and communication abilities, which makes the optimization easier, and (b) the fact that, for this kind of robot, morphological computation plays a significant role. Moreover, we discover that what matters more is the degree of collectivity the robots are required to exhibit when facing the task.

Synergizing habits and goals with variational Bayes

Behaving efficiently and flexibly is crucial for biological and artificial embodied agents. Behavior is generally classified into two types: habitual (fast but inflexible), and goal-directed (flexible but slow). While these two types of behaviors are typically considered to be managed by two distinct systems in the brain, recent studies have revealed a more sophisticated interplay between them. We introduce a theoretical framework using variational Bayesian theory, incorporating a Bayesian intention variable. Habitual behavior depends on the prior distribution of intention, computed from sensory context without goal-specification. In contrast, goal-directed behavior relies on the goal-conditioned posterior distribution of intention, inferred through variational free energy minimization. Assuming that an agent behaves using a synergized intention, our simulations in vision-based sensorimotor tasks explain the key properties of their interaction as observed in experiments. Our work suggests a fresh perspective on the neural mechanisms of habits and goals, shedding light on future research in decision making.

Emergence of integrated behaviors through direct optimization for homeostasis

Homeostasis is a self-regulatory process, wherein an organism maintains a specific internal physiological state. Homeostatic reinforcement learning (RL) is a framework recently proposed in computational neuroscience to explain animal behavior. Homeostatic RL organizes the behaviors of autonomous embodied agents according to the demands of the internal dynamics of their bodies, coupled with the external environment. Thus, it provides a basis for real-world autonomous agents, such as robots, to continually acquire and learn integrated behaviors for survival. However, prior studies have generally explored problems pertaining to limited size, as the agent must handle observations of such coupled dynamics. To overcome this restriction, we developed an advanced method to realize scaled-up homeostatic RL using deep RL. Furthermore, several rewards for homeostasis have been proposed in the literature. We identified that the reward definition that uses the difference in drive function yields the best results. We created two benchmark environments for homeostasis and performed a behavioral analysis. The analysis showed that the trained agents in each environment changed their behavior based on their internal physiological states. Finally, we extended our method to address vision using deep convolutional neural networks. The analysis of a trained agent revealed that it has visual saliency rooted in the survival environment and internal representations resulting from multimodal input.

A Social Self-Awareness Agent with Embodied Reasoning

In the realm of multi-agent systems, establishing social self-awareness in agents stands as a critical challenge within the broader scope of artificial general intelligence research. Addressing this challenge, in this paper, we draw upon Tomasello’s social self-awareness theory as a foundational framework, and employ Non-Axiomatic Reasoning System (NARS), an artificial general intelligence model, to simulate analogical and recursive reasoning methods used by socially embodied agents. In this paper, we assume that an agent can possess sensorimotor capabilities and physical common sense, it explores the process of constructing a “core self” through embodied reasoning. On this basis, we explore deeper into the construction process of social self-awareness. As a result, we clarify how agents generate common knowledge about themselves through recursive reasoning regarding the mental states of others and communication dynamics. Following this initial step, metaphors are generated utilizing analogical reasoning, intertwining public knowledge with these metaphors to facilitate mutual comprehension between humans and machines. This paper delves into the simulation of perspective representation and recursive reasoning within social embodied agents, employing NARS– an overarching artificial intelligence platform. The experimental results substantiate the viability of constructing a social self-agent within a machine framework grounded in embodied reasoning.

Enhancing efficiency and propulsion in bio-mimetic robotic fish through end-to-end deep reinforcement learning

Aquatic organisms are known for their ability to generate efficient propulsion with low energy expenditure. While existing research has sought to leverage bio-inspired structures to reduce energy costs in underwater robotics, the crucial role of control policies in enhancing efficiency has often been overlooked. In this study, we optimize the motion of a bio-mimetic robotic fish using deep reinforcement learning (DRL) to maximize propulsion efficiency and minimize energy consumption. Our novel DRL approach incorporates extended pressure perception, a transformer model processing sequences of observations, and a policy transfer scheme. Notably, significantly improved training stability and speed within our approach allow for end-to-end training of the robotic fish. This enables agiler responses to hydrodynamic environments and possesses greater optimization potential compared to pre-defined motion pattern controls. Our experiments are conducted on a serially connected rigid robotic fish in a free stream with a Reynolds number of 6000 using computational fluid dynamics simulations. The DRL-trained policies yield impressive results, demonstrating both high efficiency and propulsion. The policies also showcase the agent's embodiment, skillfully utilizing its body structure and engaging with surrounding fluid dynamics, as revealed through flow analysis. This study provides valuable insights into the bio-mimetic underwater robots optimization through DRL training, capitalizing on their structural advantages, and ultimately contributing to more efficient underwater propulsion systems.

The effect of in‐vehicle agent embodiment on drivers' perceived usability and cognitive workload: Evidence from subjective reporting, ECG, and fNIRS

AbstractTo improve the interaction between drivers and the in‐vehicle information system (IVIS), various intelligent agents, such as robot agents, virtual agents, and voice‐only agents, have been integrated into vehicles. However, it is not yet clear which type of in‐vehicle agent is best suited to the driving context. This study aims to investigate the effect of in‐vehicle agent embodiment on drivers' perceived usability and cognitive workload. In a within‐subject simulated driving experiment, 22 participants interacted with three different in‐vehicle agents (smartphone agent, robot agent, and virtual agent). Functional near‐infrared spectroscopy and electrocardiogram (ECG) were used to record prefrontal cortex activation and electrical changes associated with cardiac activity during simulated driving, respectively. The results show that the smartphone agent had the lowest perceived usability scores, oxygenated hemoglobin concentration variation (ΔHbO), and maximum ECG signal variation compared to baseline. There were no statistical differences in cognitive workload, perceived usability scores, brain area activation, and ECG signals between the robot agent and the virtual agent. The research findings demonstrate the positive effects of the anthropomorphic appearance of in‐vehicle agents on perceived usability and contribute to improving the design of in‐vehicle intelligent agents.

Combining Minds and Machines: Investigating the Fusion of Cognitive Architectures and Generative Models for General Embodied Intelligence

Cognitive architectures and generative models are two very different approaches for developing general embodied intelligence. This paper investigates their initial motivation, implementation ways, and the complementary strengths and weaknesses, and targets to fuse them into a general embodied intelligence so as to leverage strengths and complement weaknesses. Firstly, with analyzing their different application scenarios and the difficulties in further research and development, the potential synergy and possible integration strategies are explored between them. Then, by combining the strengths of cognitive architectures, which model human-like cognitive processes, and generative models, which excel in generating novel content based on learned patterns, it achieves the goal of creating embodied agents with enhanced overall capabilities. Finally, a comprehensive framework demonstrating the integration of cognitive architectures, generative models, and other AI methods to achieve general embodied intelligence is presented accompanied by an illustrative example.

To Boost Zero-Shot Generalization for Embodied Reasoning With Vision-Language Pre-Training.

Recently, there exists an increased research interest in embodied artificial intelligence (EAI), which involves an agent learning to perform a specific task when dynamically interacting with the surrounding 3D environment. There into, a new challenge is that many unseen objects may appear due to the increased number of object categories in 3D scenes. It makes developing models with strong zero-shot generalization ability to new objects necessary. Existing work tries to achieve this goal by providing embodied agents with massive high-quality human annotations closely related to the task to be learned, while it is too costly in practice. Inspired by recent advances in pre-trained models in 2D visual tasks, we attempt to boost zero-shot generalization for embodied reasoning with vision-language pre-training that can encode common sense as general prior knowledge. To further improve its performance on a specific task, we rectify the pre-trained representation through masked scene graph modeling (MSGM) in a self-supervised manner, where the task-specific knowledge is learned from iterative message passing. Our method can improve a variety of representative embodied reasoning tasks by a large margin (e.g., over 5.0% w.r.t. answer accuracy on MP3D-EQA dataset that consists of many real-world scenes with a large number of new objects during testing), and achieve the new state-of-the-art performance.

Moving beyond the lab: investigating empathy through the Empirical 5E approach.

Empathy is a complex and multifaceted phenomenon that plays a crucial role in human social interactions. Recent developments in social neuroscience have provided valuable insights into the neural underpinnings and bodily mechanisms underlying empathy. This methodology often prioritizes precision, replicability, internal validity, and confound control. However, fully understanding the complexity of empathy seems unattainable by solely relying on artificial and controlled laboratory settings, while overlooking a comprehensive view of empathy through an ecological experimental approach. In this article, we propose articulating an integrative theoretical and methodological framework based on the 5E approach (the "E"s stand for embodied, embedded, enacted, emotional, and extended perspectives of empathy), highlighting the relevance of studying empathy as an active interaction between embodied agents, embedded in a shared real-world environment. In addition, we illustrate how a novel multimodal approach including mobile brain and body imaging (MoBi) combined with phenomenological methods, and the implementation of interactive paradigms in a natural context, are adequate procedures to study empathy from the 5E approach. In doing so, we present the Empirical 5E approach (E5E) as an integrative scientific framework to bridge brain/body and phenomenological attributes in an interbody interactive setting. Progressing toward an E5E approach can be crucial to understanding empathy in accordance with the complexity of how it is experienced in the real world.

A Survey of Virtual Reality Interventions for Autistic Spectrum Disorder Therapy: A Neuroscience Perspective (P1-12.006)

<h3>Objective:</h3> This review evaluates the clinical efficacy of virtual reality as a behavioral modification tool to teach and support the social-communication needs of ASD children. <h3>Background:</h3> Autistic spectrum disorder (ASD) characterizes a taxonomy of disorders affecting social communication and interactive behaviors. Researchers in affective computing have begun to use virtual reality as a therapy tool for autistic individuals. This technology has demonstrated significant potential as our understanding of the social-communication deficits in autism increases. We conducted a literature review investigating the efficacy of virtual <i>reality</i> as an intervention for autism. <h3>Design/Methods:</h3> A search was conducted using Google Scholar, PubMed, IEEE Xplore, CDC, and the National Autism Center databases. The search keywords were: “Affective computing Autism” OR “Affective computing ASD”. <h3>Results:</h3> Five articles were included in our analysis. <h3>Emotional facial expression recognition tasks:</h3> ASD Children demonstrated significant increases in paired t-tests comparisons of pre and post-training performance in NEPSYII Affect recognition t(24)= −3.40, p=0.001 and Attention and executive function “Fluid reasoning” t(17)= −2.33, p=0.016. Significant increases were noted on the pre-training assessment for social attribution Triangles intentionality t(23) =−2.28, p=0.016. 88% of participants accurately identified the emotional expressions portrayed by a virtual avatar. <h3>Emotional regulation tasks:</h3> VRE therapy yielded results that persisted 12 to 16 months after treatment. 89% of children were able to diminish phobia-induced anxiety and four children overcame their phobias. <h3>Virtual reality receptivity in ASD:</h3> Virtual embodied agents effectively elicited continued affective and social conversational behaviors in ASD children for approximately 60 minutes. ASD children also reported <i>significantly</i> higher ratings of enjoyability t(38)= 2.26; P= 0.03)) while using virtual technology compared to experimental controls. <h3>Conclusions:</h3> Use of virtual reality as a tool for behavioral modification revealed positive application with enduring results, by demonstrating: (1) conversational spontaneity, (2) social appraisal, (3) emotional recognition, (4) anxiety and fear reduction, and (5) social-skill acquisition in autistic children. <b>Disclosure:</b> Ms. Allison has nothing to disclose. Dr. Prelock has received personal compensation in the range of $500-$4,999 for serving as an Editor, Associate Editor, or Editorial Advisory Board Member for Springer Publishing. The institution of Dr. Prelock has received research support from PCORI. The institution of Dr. Prelock has received research support from HRSA. Dr. Prelock has received publishing royalties from a publication relating to health care. Dr. Prelock has received publishing royalties from a publication relating to health care. Dr. Prelock has received publishing royalties from a publication relating to health care.

Consistent attack: Universal adversarial perturbation on embodied vision navigation

Embodied agents in vision navigation coupled with deep neural networks have attracted increasing attention. However, deep neural networks have been shown vulnerable to malicious adversarial noises, which may potentially cause catastrophic failures in Embodied Vision Navigation. Among different adversarial noises, universal adversarial perturbations (UAP), i.e., a constant image-agnostic perturbation applied on every input frame of the agent, play a critical role in Embodied Vision Navigation since they are computation-efficient and application-practical during the attack. However, existing UAP methods ignore the system dynamics of Embodied Vision Navigation and might be sub-optimal. In order to extend UAP to the sequential decision setting, we formulate the disturbed environment under the universal noise δ, as a δ-disturbed Markov Decision Process (δ-MDP). Based on the formulation, we analyze the properties of δ-MDP and propose two novel Consistent Attack methods, named Reward UAP and Trajectory UAP, for attacking Embodied agents, which consider the dynamic of the MDP and calculate universal noises by estimating the disturbed distribution and the disturbed Q function. For various victim models, our Consistent Attack can cause a significant drop in their performance in the PointGoal task in Habitat with different datasets and different scenes. Extensive experimental results indicate that there exist serious potential risks for applying Embodied Vision Navigation methods to the real world.

The promise and peril of interactive embodied agents for studying non-verbal communication: a machine learning perspective.

In face-to-face interactions, parties rapidly react and adapt to each other's words, movements and expressions. Any science of face-to-face interaction must develop approaches to hypothesize and rigorously test mechanisms that explain such interdependent behaviour. Yet conventional experimental designs often sacrifice interactivity to establish experimental control. Interactive virtual and robotic agents have been offered as a way to study true interactivity while enforcing a measure of experimental control by allowing participants to interact with realistic but carefully controlled partners. But as researchers increasingly turn to machine learning to add realism to such agents, they may unintentionally distort the very interactivity they seek to illuminate, particularly when investigating the role of non-verbal signals such as emotion or active-listening behaviours. Here I discuss some of the methodological challenges that may arise when machine learning is used to model the behaviour of interaction partners. By articulating and explicitly considering these commitments, researchers can transform 'unintentional distortions' into valuable methodological tools that yield new insights and better contextualize existing experimental findings that rely on learning technology. This article is part of a discussion meeting issue 'Face2face: advancing the science of social interaction'.

Voice Over Body? Older Adults’ Reactions to Robot and Voice Assistant Facilitators of Group Conversation

Intelligent agents have great potential as facilitators of group conversation among older adults. However, little is known about how to design agents for this purpose and user group, especially in terms of agent embodiment. To this end, we conducted a mixed methods study of older adults’ reactions to voice and body in a group conversation facilitation agent. Two agent forms with the same underlying artificial intelligence (AI) and voice system were compared: a humanoid robot and a voice assistant. One preliminary study (total n = 24) and one experimental study comparing voice and body morphologies (n = 36) were conducted with older adults and an experienced human facilitator. Findings revealed that the artificiality of the agent, regardless of its form, was beneficial for the socially uncomfortable task of conversation facilitation. Even so, talkative personality types had a poorer experience with the “bodied” robot version. Design implications and supplementary reactions, especially to agent voice, are also discussed.

What Could Models of Superorganismal Cognition Offer to Embodied AI?

Superorganisms such as ant or honeybee colonies exhibit extraordinary collective intelligence, such as an ability to identify and choose the best available nest site in an uncertain world. This collective cognition is inextricably reliant on the embodiment of individual agents, specifically their movement through space. We have recently developed models of superorganismal cognition based on a compelling analogy with techniques in Bayesian statistics, which are likewise aimed at grappling with the uncertainty and incompleteness of real data sources. These models foreground some potential lessons for the design of embodied artificial intelligences, such as robot swarms. For example, the spatial distribution of independently judging agents can convey valuable information, relaxing expectations that regular inter-agent (‘inter-neuronal’) communication is necessary for cognition, which points to the potential of minimal field swarm robotics. Meanwhile, the importance of individual heterogeneity to effective and resilient collective cognition in biology suggests great potential in this area for engineering.

A review of platforms for simulating embodied agents in 3D virtual environments

A review of platforms for simulating embodied agents in 3D virtual environments

The Case for Negotiation Robots in simulated workplace negotiations A Theoretical Approach

Job negotiations are an anxiety-inducing and highly important form of social interaction. Most people are not sufficiently trained in negotiation strategies and often do not receive job offers that are conducive to financial stability. With the rise of virtual agents and robotics that are capable of modeling social interactions, a significant research effort has been established to create realistic simulations of negotiations. While the bulk of this research has focused on virtual agents as a medium for simulated negotiations, we propose that embodied agents can be utilized to model the inherent nuances in human interaction. In this paper, we propose a study that aims to evaluate the effectiveness of embodied agents compared to virtual agents in simulated negotiations with human participants.

Evolving Modularity in Soft Robots Through an Embodied and Self-Organizing Neural Controller.

Modularity is a desirable property for embodied agents, as it could foster their suitability to different domains by disassembling them into transferable modules that can be reassembled differently. We focus on a class of embodied agents known as voxel-based soft robots (VSRs). They are aggregations of elastic blocks of soft material; as such, their morphologies are intrinsically modular. Nevertheless, controllers used until now for VSRs act as abstract, disembodied processing units: Disassembling such controllers for the purpose of module transferability is a challenging problem. Thus, the full potential of modularity for VSRs still remains untapped. In this work, we propose a novel self-organizing, embodied neural controller for VSRs. We optimize it for a given task and morphology by means of evolutionary computation: While evolving, the controller spreads across the VSR morphology in a way that permits emergence of modularity. We experimentally investigate whether such a controller (i) is effective and (ii) allows tuning of its degree of modularity, and with what kind of impact. To this end, we consider the task of locomotion on rugged terrains and evolve controllers for two morphologies. Our experiments confirm that our self-organizing, embodied controller is indeed effective. Moreover, by mimicking the structural modularity observed in biological neural networks, different levels of modularity can be achieved. Our findings suggest that the self-organization of modularity could be the basis for an automatic pipeline for assembling, disassembling, and reassembling embodied agents.

Braitenberg Vehicles as Developmental Neurosimulation.

Connecting brain and behavior is a longstanding issue in the areas of behavioral science, artificial intelligence, and neurobiology. As is standard among models of artificial and biological neural networks, an analogue of the fully mature brain is presented as a blank slate. However, this does not consider the realities of biological development and developmental learning. Our purpose is to model the development of an artificial organism that exhibits complex behaviors. We introduce three alternate approaches to demonstrate how developmental embodied agents can be implemented. The resulting developmental Braitenberg vehicles (dBVs) will generate behaviors ranging from stimulus responses to group behavior that resembles collective motion. We will situate this work in the domain of artificial brain networks along with broader themes such as embodied cognition, feedback, and emergence. Our perspective is exemplified by three software instantiations that demonstrate how a BV-genetic algorithm hybrid model, a multisensory Hebbian learning model, and multi-agent approaches can be used to approach BV development. We introduce use cases such as optimized spatial cognition (vehicle-genetic algorithm hybrid model), hinges connecting behavioral and neural models (multisensory Hebbian learning model), and cumulative classification (multi-agent approaches). In conclusion, we consider future applications of the developmental neurosimulation approach.