Human-agent Systems Research Articles

Empirical software engineering for evaluating adaptive task-oriented personal assistants: a case study in human/machine event extraction and coding

This paper concerns methodology for evaluating task-oriented personal assistants, where users perform a complex task that has objective success criteria, independent of personal preferences, and the agent provides suggestions to help users repeatedly perform the task consistently and accurately. We develop a systematic approach to evaluating task-oriented personal assistants in normal contexts of use through extending the methodology of empirical software engineering to evaluate effectiveness, efficiency and satisfaction. The approach allows the evaluation of both the human-agent system and of the personal assistant using data obtained by observations of user and system behaviour. A key element of our approach is to define empirically observable conditions that separate the learning period, when users and the agent are learning to perform the task, from the evaluation period, when performance benefits are measured. The methodology is illustrated using the example of a system for users to extract, annotate and code events from news articles.

Reciprocity and Its Neurological Correlates in Human-Agent Cooperation

Reciprocal cooperation is prevalent in human society. Understanding human reciprocal cooperation in human-agent interaction can help design human-agent systems that promote cooperation and joint performance. Studies have found that people reciprocate cooperative behavior when interacting with computer agents in social dilemma games. However, few studies have investigated human reciprocal cooperation with agents in complex dynamic environments. This article examines the behavioral and neurological patterns of reciprocal cooperation in a hospital management microworld experiment. The participants ( n = 30) work with both high- and low-cooperation computer agents to share resources to cope with dynamic demands. Participants’ resource sharing behaviors were recorded and their prefrontal cortex (PFC) activation was measured using functional near-infrared spectroscopy (fNIRS) technology. Similar to previous studies conducted with participants in the United States, results demonstrate that participants in China showed reciprocal cooperation behaviors with the agents. Specifically, participants share more resources and achieve higher performance when working with a high-cooperation agent than with a low-cooperation agent. A high activation level is detected in the right dorsolateral PFC when working with a high-cooperation agent. Other PFC activation patterns imply that cooperation could be unnecessarily mentally taxing in certain situations. These findings suggest that human cooperativeness in human–agent systems can be calibrated by an agent's cooperation behavior. System designers should design for appropriate cooperativeness and avoid the inefficient use of system resources. Neurological measures could be a useful tool to investigate the mental process in human-agent cooperation

Conceptual Design of a Generic Nodal Abstraction (GNA) for a Human-Agent Collaboration Systems

The issues of modelling software agents in collaborative systems present some problems and difficulties caused by the diversity of tasks and the procedures attached to them. Consequently, such diversity poses a critical challenge to implement scheduled tasks. In this paper, we propose collaborative human-agent system in which a software agent collaborates and cooperates with its human counterpart in several job positions of an organization. The goal of the GNA approach is to improve collaboration between humans when performing tasks within a workflow process. The GNA is built based on the concept of a node which comprises of a human, agents, and their shared functions. This concept espouses the notion that agents could be deployed to assist humans in many common workflow processes and to handle mundane tasks. Furthermore, a multi-agent system is advocated as an application or platform for managing distributed data. Specifically, this paper describes a conceptual design of a generic approach based on the notion of nodes to construct a Generic Nodal ion (GNA) approach that is deployable as a modeling and simulation platform for organizational processes.

Explainability in human–agent systems

This paper presents a taxonomy of explainability in human–agent systems. We consider fundamental questions about the Why, Who, What, When and How of explainability. First, we define explainability, and its relationship to the related terms of interpretability, transparency, explicitness, and faithfulness. These definitions allow us to answer why explainability is needed in the system, whom it is geared to and what explanations can be generated to meet this need. We then consider when the user should be presented with this information. Last, we consider how objective and subjective measures can be used to evaluate the entire system. This last question is the most encompassing as it will need to evaluate all other issues regarding explainability.

A hybrid approach to decision making and information fusion: Combining humans and artificial agents

This paper argues that hybrid human–agent systems can support powerful solutions to relevant problems such as Environmental Crisis management. However, it shows that such solutions require comprehensive approaches covering different aspects of data processing, model construction and the usage. In particular, the solutions (i) must be able to cope with complex correlations (as different data sources are used) and processing of large amounts of data, (ii) must be robust against modeling imperfections and (iii) human–machine interaction (HMI) approaches must facilitate human use of crisis management tools and reduce the likelihood of miscommunication.In this paper the relevant problem is an environmental protection application involving the detection and tracking of gases in case of chemical spills in an urban area. We show that a combination of Bayesian Networks, agent paradigm and systematic approaches to implementing HMI, support effective and robust solutions. To better integrate human information and demonstrate the usefulness of user generated crisis response information we developed a social media harvesting interface based on data from Twitter tweets and a visual interface to facilitate human smell classification.

Cooperation in Human-Agent Systems to Support Resilience: A Microworld Experiment.

This study uses a dyadic approach to understand human-agent cooperation and system resilience. Increasingly capable technology fundamentally changes human-machine relationships. Rather than reliance on or compliance with more or less reliable automation, we investigate interaction strategies with more or less cooperative agents. A joint-task microworld scenario was developed to explore the effects of agent cooperation on participant cooperation and system resilience. To assess the effects of agent cooperation on participant cooperation, 36 people coordinated with a more or less cooperative agent by requesting resources and responding to requests for resources in a dynamic task environment. Another 36 people were recruited to assess effects following a perturbation in their own hospital. Experiment 1 shows people reciprocated the cooperative behaviors of the agents; a low-cooperation agent led to less effective interactions and less resource sharing, whereas a high-cooperation agent led to more effective interactions and greater resource sharing. Experiment 2 shows that an initial fast-tempo perturbation undermined proactive cooperation-people tended to not request resources. However, the initial fast tempo had little effect on reactive cooperation-people tended to accept resource requests according to cooperation level. This study complements the supervisory control perspective of human-automation interaction by considering interdependence and cooperation rather than the more common focus on reliability and reliance. The cooperativeness of automated agents can influence the cooperativeness of human agents. Design and evaluation for resilience in teams involving increasingly autonomous agents should consider the cooperative behaviors of these agents.