Human-agent Collaboration Research Articles (Page 1)

Abstract - The rapid development of agentic AI, where autonomous agents act instead of humans need to align with human values to attain their objectives, despite initially having limited knowledge of those values.Using traditional reinforcement learning in agentic AI may result in incorrectly formulated reward functions resulting in unsafe behaviours. Furthermore RL agents encounter difficulties in sampling, reward misalignment, and lack of cross-domain generalization. In this paper, we introduce Bayesian Cooperative Inverse Reinforcement Learning (Bayesian CIRL). This is a new method that uses a latent variable with a probabilistic belief about the distribution of the human reward function. Unlike the traditional IRL, Bayesian CIRL treats value alignment as a cooperative game, where the agent updates its beliefs by observing human actions and also plans its own actions to maximize the uncertain joint reward. This approach allows adaptive cooperation through active learning. It also transfer the learned behaviour to reduce uncertainty over human inclinations. Experimental tests show that Bayesian CIRL offers more robust and accurate value alignment than standard IRL algorithms. It handles ambiguity better and allows for reliable interaction between humans and agents. This framework provides a clear way to introduce agentic AI systems that align with human ethics and societal expectations. Key Words: Agentic AI, Reinforcement Learning, Inverse Reinforcement Learning, Cooperative Inverse Reinforcement Learning, Bayesian CIRL

Visual analytics (VA) requires analysts to iteratively propose analysis tasks based on observations and execute tasks by creating visualizations and interactive exploration to gain insights. This process demands skills in programming, data processing, and visualization tools, highlighting the need for a more intelligent, streamlined VA approach. Large language models (LLMs) have recently been developed as agents to handle various tasks with dynamic planning and tool-using capabilities, offering the potential to enhance the efficiency and versatility of VA. We propose LightVA, a lightweight VA framework that supports task decomposition, data analysis, and interactive exploration through human-agent collaboration. Our method is designed to help users progressively translate high-level analytical goals into low-level tasks, producing visualizations and deriving insights. Specifically, we introduce an LLM agent-based task planning and execution strategy, employing a recursive process involving a planner, executor, and controller. The planner is responsible for recommending and decomposing tasks, the executor handles task execution, including data analysis, visualization generation and multi-view composition, and the controller coordinates the interaction between the planner and executor. Building on the framework, we develop a system with a hybrid user interface that includes a task flow diagram for monitoring and managing the task planning process, a visualization panel for interactive data exploration, and a chat view for guiding the model through natural language instructions. We examine the effectiveness of our method through a usage scenario and an expert study.

Collaborating in real-life situations rarely follows predefined roles or plans, but is established on the fly and flexibly coordinated by the interacting agents. We introduce the notion of fluid collaboration (FC), marked by frequent changes of the tasks partners assume or the resources they consume in response to varying requirements or affordances of the environment, tasks, or other agents. FC thus necessitates dynamic, action-oriented Theory of Mind reasoning to enable agents to continuously infer and adapt to others' intentions and beliefs in real-time. In this paper, we discuss how FC can be enabled in human-agent collaboration. We introduce Cooperative Cuisine, an interactive environment inspired by the game Overcooked! that facilitates human-human and human-agent collaboration in dynamic settings. We report results of an empirical study on human-human collaboration in CoCu, showing how FC can be measured empirically and that humans naturally engage in dynamically established collaboration patterns with minimal explicit communication and relying on efficient mentalizing. We then present an approach to develop artificial agents that can effectively participate in FC. Specifically, we argue for a model of dynamic mentalizing under computational constraints and integrated with action planning. We present first steps in this direction by addressing resource-rational and action-driven ToM reasoning.

Over the past few decades, autonomous agents have undergone tremendous evolution, moving from rule based systems to highly adaptive, learning-driven architectures. These autonomously perceivable, reasoning, and acting agents have found use in robotics, healthcare, finance, and other fields. This survey provides a comprehensive overview of the evolution of autonomous agents, highlighting key technological advancements, emerging trends, and persistent challenges. We explore the role of deep reinforcement learning, multi-agent systems, neuro symbolic AI, and edge computing in enhancing agent autonomy. Additionally, we discuss critical challenges such as generalization, safety, scalability, and ethical considerations. Finally, we outline future research directions, emphasizing the need for robust generalization techniques, improved human-agent collaboration, and the integration of quantum computing and self supervised learning. This study acts as an important tool for researchers and practitioners aiming to comprehend the present scenario and prospective of autonomous agents.

The complexity of modern IT operations, driven by cloud adoption, microservices, and DevOps, challenges traditional management, causing inefficiencies and reactive incident resolution. This paper proposes Agentic AI as a transformative paradigm for truly autonomous IT operations, progressing beyond automation to intelligent and self-governing systems. This paper presents a framework for Agentic AI in IT operations, highlighting key components: Perception, knowledge and memory, decision, and Action, along with the importance of multi-agent orchestration and human-agent collaboration. We outline key design principles for robust autonomous systems, including progressive autonomy, self-healing, observability and explainability, scalability and elasticity, security by design, and continuous learning. Implementation strategies highlight cloud-native approaches and integration with existing IT ecosystems. We acknowledge challenges such as building trust, managing integration complexity, and addressing ethics, while identifying future research directions like human-AI teaming. This paper offers a roadmap for enhancing automation, improving resilience, and optimizing efficiency, enabling organizations to navigate digital transformation with agility.

Regulatory Compliance Agents powered by artificial intelligence are transforming how enterprises navigate increasingly complex regulatory environments. These intelligent systems autonomously monitor, interpret, and implement regulatory requirements across multiple jurisdictions, offering unprecedented efficiency improvements in compliance management. By leveraging advanced natural language processing and machine learning technologies, these agents demonstrate remarkable capabilities in processing regulatory documents, identifying relevant changes, and translating abstract requirements into concrete operational controls. Implementation across diverse industries including financial services, healthcare, energy, and pharmaceuticals shows consistent benefits in reducing compliance costs, improving accuracy, and accelerating regulatory response times. While offering transformative potential, these solutions face challenges related to interpretive accuracy, regulatory acceptance, and technical integration. Organizations implementing these systems must carefully address governance frameworks and human-agent collaboration models to maximize benefits while maintaining appropriate oversight. As regulatory landscapes continue to evolve, AI-powered compliance agents represent not merely technological innovations but strategic assets that fundamentally reshape how organizations approach compliance management, enabling more proactive, efficient, and comprehensive regulatory navigation in an increasingly regulated global economy.

The impact of individual AI proficiency on human–agent collaboration: Higher sensitivity to discern the comprehension ability of intelligent agents for users with higher AI proficiency levels

Background: Healthcare robotics needs context-aware policy-compliant reasoning to achieve safe human-agent collaboration. The current ontologies fail to provide healthcare-relevant information and flexible semantic enforcement systems. Methods: HERON represents a modular upper ontology which enables healthcare robotic systems to communicate and collaborate while ensuring safety during operations. The system enables domain-specific instantiations through SPARQL queries and SHACL-based constraint validation to perform context-driven logic. The system models robotic task interactions through simulated eldercare and diagnostic and surgical support scenarios which follow ethical and regulatory standards. Results: The validation tests demonstrated HERON's capacity to enable safe and explainable autonomous operations in changing environments. The semantic constraints enforced proper eligibility for roles and privacy conditions and policy override functionality during agent task execution. The HERON system demonstrated compatibility with healthcare IT systems and demonstrated adaptability to the GDPR and other policy frameworks. Conclusions: The semantically rich framework of HERON establishes an interoperable foundation for healthcare robotics. The system architecture maintains an open design which enables HL7/FHIR standard integration and robotic middleware compatibility. HERON demonstrates superior healthcare-specific capabilities through its evaluation against SUMO HL7 and MIMO. The future research will focus on optimizing HERON for low-resource clinical environments while extending its applications to remote care emergency triage and adaptive human-robot collaboration.

Goal recognition (GR) involves inferring an agent's unobserved goal from a sequence of observations. This is a critical problem in AI with diverse applications. Traditionally, GR has been addressed using 'inference to the best explanation' or abduction, where hypotheses about the agent's goals are generated as the most plausible explanations for observed behavior. Alternatively, some approaches enhance interpretability by ensuring that an agent's behavior aligns with an observer's expectations or by making the reasoning behind decisions more transparent. In this work, we tackle a different challenge: explaining the GR process in a way that is comprehensible to humans. We introduce and evaluate an explainable model for goal recognition (GR) agents, grounded in the theoretical framework and cognitive processes underlying human behavior explanation. Drawing on insights from two human-agent studies, we propose a conceptual framework for human-centered explanations of GR. Using this framework, we develop the eXplainable Goal Recognition (XGR) model, which generates explanations for both why and why not questions. We evaluate the model computationally across eight GR benchmarks and through three user studies. The first study assesses the efficiency of generating human-like explanations within the Sokoban game domain, the second examines perceived explainability in the same domain, and the third evaluates the model's effectiveness in aiding decision-making in illegal fishing detection. Results demonstrate that the XGR model significantly enhances user understanding, trust, and decision-making compared to baseline models, underscoring its potential to improve human-agent collaboration.

Efficient human-agent collaboration requires understanding each other’s capabilities and establishing appropriate reliance. My thesis focuses on optimizing performance in mixed-initiative settings, where humans and agents dynamically contribute to decisions and actions. I first explore key factors shaping human reliance on decision-support agents, then examine how agents can model this reliance to initiate actions. My proposed work aims to enable agents to jointly provide decision and action support in multi-objective tasks, using bi-directional communication to enhance collaboration.

This study investigated whether bidirectional transparency, compared to agent-to-human transparency, improved human–agent collaboration. Additionally, we examined the optimal transparency levels for both humans and agents. We assessed the impact of transparency direction and level on various metrics of a human-agent team, including performance, trust, satisfaction, perceived agent’s teaming skills, and mental workload. A total of 30 participants engaged in a human-agent collaborative game in a within-subject experiment with five conditions: a 2 (transparency directions: agent-to-human transparency vs. bidirectional transparency) × 2 (transparency levels: reasoning transparency vs. reasoning + projection transparency) factorial design, plus an additional action transparency condition as a control condition. The findings indicated that bidirectional transparency improved task performance without increasing the mental workload. This study recommends a bidirectional transparency mechanism, in which the agent provides transparency to humans regarding its reasoning and predictions, whereas humans offer transparency to the agent regarding their reasoning. Practitioner Summary: This study highlights the importance of bidirectional transparency in human-agent collaboration, demonstrating its effectiveness in enhancing task performance without increasing mental workload. It recommends implementing a mechanism where both humans and agents share transparency information, optimising collaboration outcomes.

IntroductionHuman-centric artificial intelligence (HCAI) focuses on systems that support and collaborate with humans to achieve their goals. To better understand how collaboration develops in human-AI teaming, further exploration grounded in a theoretical model is needed. Tuckman's model describes how team development among humans evolves by transitioning through the stages of forming, storming, norming, performing, and adjourning. The purpose of this pilot study was to explore transitions between the first three stages in a collaborative task involving a human and a human-centric agent.MethodThe collaborative task was selected based on commonly performed tasks in a therapeutic healthcare context. It involved planning activities for the upcoming week to achieve health-related goals. A calendar application served as a tool for this task. This application embedded a collaborative agent designed to interact with humans following Tuckman's stages of team development. Eight participants completed the collaborative calendar planning task, followed by a semi-structured interview. Interviews were transcribed and analyzed using inductive content analysis.ResultsThe results revealed that the participants initiated the storming stage in most cases (n = 7/8) and that the agent initiated the norming stage in most cases (n = 5/8). Additionally, three main categories emerged from the content analyses of the interviews related to participants' transition through team development stages: (i) participants' experiences of Tuckman's first three stages of team development; (ii) their reactions to the agent's behavior in the three stages; and (iii) factors important to the participants to team up with a collaborative agent.ConclusionResults suggest ways to further personalize the agent to contribute to human-agent teamwork. In addition, this study revealed the need to further examine the integration of explicit conflict management into human-agent collaboration for human-agent teamwork.

Training reinforcement learning (RL) agents to achieve desired goals while also acting morally is a challenging problem. Transformer-based language models (LMs) have shown some promise in moral awareness, but their use in different contexts is problematic because of the complexity and implicitness of human morality. In this paper, we build on text-based games, which are challenging environments for current RL agents, and propose the HuMAL (Human-guided Morality Awareness Learning) algorithm, which adaptively learns personal values through human-agent collaboration with minimal manual feedback. We evaluate HuMAL on the Jiminy Cricket benchmark, a set of text-based games with various scenes and dense morality annotations, using both simulated and actual human feedback. The experimental results demonstrate that with a small amount of human feedback, HuMAL can improve task performance and reduce immoral behavior in a variety of games and is adaptable to different personal values.

Appropriately calibrated human trust is essential for successful Human-Agent collaboration. Probabilistic frameworks using a partially observable Markov decision process (POMDP) have been previously employed to model the trust dynamics of human behavior, optimising the outcomes of a task completed with a collaborative recommender system. A POMDP model utilising signal detection theory to account for latent user trust is presented, with the model working to calibrate user trust via the implementation of three distinct agent features: disclaimer message, request for additional information, and no additional feature. A simulation experiment is run to investigate the efficacy of the proposed POMDP model compared against a random feature model and a control model. Evidence demonstrates that the proposed POMDP model can appropriately adapt agent features in-task based on human trust belief estimates in order to achieve trust calibration. Specifically, task accuracy is highest with the POMDP model, followed by the control and then the random model. This emphasises the importance of trust calibration, as agents that lack considered design to implement features in an appropriate way can be more detrimental to task outcome compared to an agent with no additional features.

Optimal performance of collaborative tasks requires consideration of the interactions between intelligent agents and their human counterparts. The functionality and success of these agents lie in their ability to maintain user trust; with too much or too little trust leading to over-reliance and under-utilisation, respectively. This problem highlights the need for an appropriate trust calibration methodology with an ability to vary user trust and decision making in-task. An online experiment was run to investigate whether stimulus difficulty and the implementation of agent features by a collaborative recommender system interact to influence user perception, trust and decision making. Agent features are changes to the Human-Agent interface and interaction style, and include presentation of a disclaimer message, a request for more information from the user and no additional feature. Signal detection theory is utilised to interpret decision making, with this applied to assess decision making on the task, as well as with the collaborative agent. The results demonstrate that decision change occurs more for hard stimuli, with participants choosing to change their initial decision across all features to follow the agent recommendation. Furthermore, agent features can be utilised to mediate user decision making and trust in-task, though the direction and extent of this influence is dependent on the implemented feature and difficulty of the task. The results emphasise the complexity of user trust in Human-Agent collaboration, highlighting the importance of considering task context in the wider perspective of trust calibration.

When a human and machine collaborate on a shared task, ambiguous events might occur that could be perceived as an error by the human partner. In such events, spontaneous error-related potentials (ErrPs) are evoked in the human brain. Knowing whom the human perceived as responsible for the error would help a machine in co-adaptation and shared control paradigms to better adapt to human preferences. Therefore, we ask whether self- and agent-related errors evoke different ErrPs. Eleven subjects participated in an electroencephalography human-agent collaboration experiment with a collaborative trajectory-following task on two collaboration levels, where movement errors occurred as trajectory deviations. Independently of the collaboration level, we observed a higher amplitude of the responses on the midline central Cz electrode for self-related errors compared to observed errors made by the agent. On average, Support Vector Machines classified self- and agent-related errors with 72.64% accuracy using subject-specific features. These results demonstrate that ErrPs can tell if a person relates an error to themselves or an external autonomous agent during collaboration. Thus, the collaborative machine will receive more informed feedback for the error attribution that allows appropriate error identification, a possibility for correction, and avoidance in future actions.

Despite various attempts to prevent food waste and motivate conscious food handling, household members find it difficult to correctly assess the edibility of food. With the rise of ambient voice assistants, we did a design case study to support households’ in situ decision-making process in collaboration with our voice agent prototype, Fischer Fritz. Therefore, we conducted 15 contextual inquiries to understand food practices at home. Furthermore, we interviewed six fish experts to inform the design of our voice agent on how to guide consumers and teach food literacy. Finally, we created a prototype and discussed with 15 consumers its impact and capability to convey embodied knowledge to the human that is engaged as sensor. Our design research goes beyond current Human-Food Interaction automation approaches by emphasizing the human-food relationship in technology design and demonstrating future complementary human-agent collaboration with the aim to increase humans’ competence to sense, think, and act.

The human-agent collaboration (HAC) is a prospective research topic, whose great applications and future scenarios have attracted vast attention. It is very important to understand the design process of the HAC system (HACS). Inspired by the systematic analysis framework presented in Part I of this dual publication, this article proposes a normalized two-phase procedure, namely, GET-MAN, for the top-level design of HACS from the perspective of system engineering. The two-phase design procedure can produce a coherent and well-running HACS by sophisticatedly and properly determining the six elements of the HACS and their influences. In the verification phase of GET-MAN, by applying the formalized HACS framework proposed in Part I, a formal model can be constructed to look ahead (predict) and back (explain) at potential faults in the candidate HACS. An example of the HACS design for target searching is employed to illustrate the use of the GET-MAN design procedure. The potential challenges and future research directions are discussed in the light of the GET-MAN design procedure. The systematic analysis framework, Part I, as well as the GET-MAN design procedure, Part II, can serve as common guidance and reference for analyzing and developing various HACSs.

Increasing agent transparency is an ongoing challenge for Human-Agent Collaboration (HAC). Chen et al. proposed the three level SAT framework to improve Agent Transparency and users’ Situational Awareness (SA) by informing about (1) what the agent is doing, (2) why the agent is doing it and (3) what the agent will do next. Explanations can be descriptive (informing the user decision-making process) or prescriptive (guiding the user toward a pre-determined choice). To study these differences, we conducted a 3 (SA level) x 2 (explanation types) online between-group user experiment (n=180) where we designed six visual explanations and tested their impact on task performance, reliance, reported trust, cognitive load and situational awareness in a goal-oriented HAC interactive task. We found that SA level 1 explanations led to better task performance, while SA level 2 explanations increased trust. Moreover, descriptive explanations had a more positive impact on participants compared to prescriptive explanations.

Achieving successful human–agent collaboration in the context of smart environments requires the modeling of human behavior for predicting people’s decisions. The goal of the current study was to utilize the TBR and the Alpha band as electrophysiological features that will discriminate between different tasks, each associated with a different depth of reasoning. To that end, we monitored the modulations of the TBR and Alpha, while participants were engaged in performing two cognitive tasks: picking and coordination. In the picking condition (low depth of processing), participants were requested to freely choose a single word out of a string of four words. In the coordination condition (high depth of processing), participants were asked to try and select the same word as an unknown partner that was assigned to them. We performed two types of analyses, one that considers the time factor (i.e., observing dynamic changes across trials) and the other that does not. When the temporal factor was not considered, only Beta was sensitive to the difference between picking and coordination. However, when the temporal factor was included, a transition occurred between cognitive effort and fatigue in the middle stage of the experiment. These results highlight the importance of monitoring the electrophysiological indices, as different factors such as fatigue might affect the instantaneous relative weight of intuitive and deliberate modes of reasoning. Thus, monitoring the response of the human–agent across time in human–agent interactions might turn out to be crucial for smooth coordination in the context of human–computer interaction.