Agents In Dynamic Environments Research Articles

Preserving paradata for accountability of semi-autonomous AI agents in dynamic environments: An archival perspective

This paper proposes the category of real-time artificial intelligence (AI) systems as applications of computerized control systems in dynamic, time-constrained contexts normally managed by human intelligence. Noting the accountability challenges which these systems introduce, the paper posits the need for robust documentation and records capacities within these systems. The paper surveys four real-time AI systems with significant records needs: autonomous vehicles, online content targeting systems, mixed-reality tools for surgical contexts, and digital twin systems in airport facilities management. The paper identifies paradata, or the data leading up to an output in a system's operation, as a key data category necessitating preservation for full transparency in the records generated by these systems. Paradata is defined as “information about the procedure(s) and tools used to create and process information resources, along with information about the persons carrying out those procedures.” Paradata uncovers opaque technological processes underlying the production of other datasets and at a granular level must be identified and preserved to delineate the boundaries between human and system agency in semi-autonomous systems. With a basis in control theory, the paper finally offers a framework for assessing the functions of real-time AI systems' operations and their documentation and records needs.

Security and Safety-Critical Learning-Based Collaborative Control for Multiagent Systems.

This article presents a novel learning-based collaborative control framework to ensure communication security and formation safety of nonlinear multiagent systems (MASs) subject to denial-of-service (DoS) attacks, model uncertainties, and barriers in environments. The framework has a distributed and decoupled design at the cyber-layer and the physical layer. A resilient control Lyapunov function-quadratic programming (RCLF-QP)-based observer is first proposed to achieve secure reference state estimation under DoS attacks at the cyber-layer. Based on deep reinforcement learning (RL) and control barrier function (CBF), a safety-critical formation controller is designed at the physical layer to ensure safe collaborations between uncertain agents in dynamic environments. The framework is applied to autonomous vehicles for area scanning formations with barriers in environments. The comparative experimental results demonstrate that the proposed framework can effectively improve the resilience and robustness of the system.

Learning in Cooperative Multiagent Systems Using Cognitive and Machine Models

Developing effective multi-agent systems (MASs) is critical for many applications requiring collaboration and coordination with humans. Despite the rapid advance of multi-agent deep reinforcement learning (MADRL) in cooperative MASs, one of the major challenges that remain is the simultaneous learning and interaction of independent agents in dynamic environments in the presence of stochastic rewards. State-of-the-art MADRL models struggle to perform well in Coordinated Multi-agent Object Transportation Problems (CMOTPs) wherein agents must coordinate with each other and learn from stochastic rewards. In contrast, humans often learn rapidly to adapt to non-stationary environments that require coordination among people. In this article, motivated by the demonstrated ability of cognitive models based on Instance-based Learning Theory (IBLT) to capture human decisions in many dynamic decision-making tasks, we propose three variants of multi-agent IBL models (MAIBLs). The idea of these MAIBL algorithms is to combine the cognitive mechanisms of IBLT and the techniques of MADRL models to deal with coordination MASs in stochastic environments from the perspective of independent learners. We demonstrate that the MAIBL models exhibit faster learning and achieve better coordination in a dynamic CMOTP task with various settings of stochastic rewards compared to current MADRL models. We discuss the benefits of integrating cognitive insights into MADRL models.

Online Risk-Bounded Motion Planning for Autonomous Vehicles in Dynamic Environments

A crucial challenge to efficient and robust motion planning for autonomous vehicles is understanding the intentions of the surrounding agents. Ignoring the intentions of the other agents in dynamic environments can lead to risky or overconservative plans. In this work, we model the motion planning problem as a partially observable Markov decision process (POMDP) and propose an online system that combines an intent recognition algorithm and a POMDP solver to generate risk-bounded plans for the ego vehicle navigating with a number of dynamic agent vehicles. The intent recognition algorithm predicts the probabilistic hybrid motion states of each agent vehicle over a finite horizon using Bayesian filtering and a library of pre-learned maneuver motion models. We update the POMDP model with the intent recognition results in real time and solve it using a heuristic search algorithm which produces policies with upper-bound guarantees on the probability of near colliding with other dynamic agents. We demonstrate that our system is able to generate better motion plans in terms of efficiency and safety in a number of challenging environments including unprotected intersection left turns and lane changes as compared to the baseline methods.

Behavioural Plasticity Can Help Evolving Agents in Dynamic Environments but at the Cost of Volatility

Neural networks have been widely used in agent learning architectures; however, learnings for one task might nullify learnings for another. Behavioural plasticity enables humans and animals alike to respond to environmental changes without degrading learned knowledge; this can be achieved by regulating behaviour with neuromodulation—a biological process found in the brain. We demonstrate that by modulating activity-propagating signals, neurally trained agents evolving to solve tasks in dynamic environments that are prone to change can expect a significantly higher fitness than non-modulatory agents and also achieve their goals more often. Further, we show that while behavioural plasticity can help agents to achieve goals in these variable environments, this ability to overcome environmental changes with greater success comes at the cost of highly volatile evolution.

CARL

Motion synthesis in a dynamic environment has been a long-standing problem for character animation. Methods using motion capture data tend to scale poorly in complex environments because of their larger capturing and labeling requirement. Physics-based controllers are effective in this regard, albeit less controllable. In this paper, we present CARL, a quadruped agent that can be controlled with high-level directives and react naturally to dynamic environments. Starting with an agent that can imitate individual animation clips, we use Generative Adversarial Networks to adapt high-level controls, such as speed and heading, to action distributions that correspond to the original animations. Further fine-tuning through the deep reinforcement learning enables the agent to recover from unseen external perturbations while producing smooth transitions. It then becomes straightforward to create autonomous agents in dynamic environments by adding navigation modules over the entire process. We evaluate our approach by measuring the agent's ability to follow user control and provide a visual analysis of the generated motion to show its effectiveness.

Robot Navigation among External Autonomous Agents through Deep Reinforcement Learning using Graph Attention Network

Abstract Finding collision-free and efficient paths in an uncertain dynamic environment is a challenge for robot navigation tasks, especially when there are external autonomous agents that also have decision-making abilities in the same environment. This paper develops a novel method based on DRL with graph attention network (GAT) to solve the problem of robot navigation among external autonomous agents (other agents). Specifically, GAT is adopted to describe the robot and other agents as a specific graph, and extract the spatial structural influence features of other agents on the robot from the graph. Multi-head attention mechanism is utilized to calculate the weights of interactions between the robot and other agents. This GAT uses observations of an arbitrary number of other agents in dynamic environments. Furthermore, the proposed method is combined with optimal reciprocal collision avoidance to improve its safety in new environments. Various simulations demonstrate that our method has good performance and robustness in different environments.

A contract-net model for foraging agents in dynamic environment

A contract-net model for foraging agents in dynamic environment

Adaptive goal selection for agents in dynamic environments

In psychology, goal-setting theory, which has been studied by psychologists for over 35 years, reveals that goals play significant roles in incentive, action and performance for human beings. Based on this theory, a goal net model has been proposed to design intelligent agents that can be viewed as a soft copy of human being somehow. The goal net model has been successfully applied in many agents, specially, non-player-character agents in computer games. Such an agent selects the optimal solution in all possible solutions found by using a recursive algorithm. However, if a goal net is very complex, the time of selection could be too long for the agent to respond quickly when the agent needs to re-select a new solution against the world’s change. Moreover, in some dynamic environments, it is impossible to know the exact outcome of choosing a solution in advance, and so the possible solutions cannot be evaluated precisely. Thus, to address the problem, this paper applies learning algorithm into goal selection in dynamic environments. More specifically, we first develop a reorganization algorithm that can convert a goal net to its equivalent counterpart that a Q-learning algorithm can operate on; then, we define the key component of Q-learning, reward function, according to the feature of goal nets; and finally lots of experiments are conducted to show that, in dynamic environments, the agent with the learning algorithm significantly outperforms the one with the recursive searching algorithm. Therefore, our work suggests an agent model that can effectively be applied in dynamic time-sensitive domain, like computer games and the P2P systems of online movie watching.

Towards adaptive programming

Current programming languages and software engineering paradigms are proving insufficient for building intelligent multi-agent systems--such as interactive games and narratives--where developers are called upon to write increasingly complex behavior for agents in dynamic environments. A promising solution is to build adaptive systems; that is, to develop software written specifically to adapt to its environment by changing its behavior in response to what it observes in the world. In this paper we describe a new programming language, An Adaptive Behavior Language (A2BL), that implements adaptive programming primitives to support partial programming, a paradigm in which a programmer need only specify the details of behavior known at code-writing time, leaving the run-time system to learn the rest. Partial programming enables programmers to more easily encode software agents that are difficult to write in existing languages that do not offer language-level support for adaptivity. We motivate the use of partial programming with an example agent coded in a cutting-edge, but non-adaptive agent programming language (ABL), and show how A2BL can encode the same agent much more naturally.

A general framework for parallel BDI agents in dynamic environments

In this paper, a general framework for the parallel BDI model suitable for dynamic environments is proposed. It is a parallel agent architecture that supports the following agent abilities at architecture level: (1) the ability to monitor the environment at all times and respond to emergencies timely; (2) the ability to reconsider and re-schedule goals, intentions and actions in reaction to unexpected or new information; (3) the ability to perform multiple actions at once; (4) the ability to perceive, deliberate and act simultaneously; (5) the ability to prioritize the deliberations and intention executions. We define the functions and the operations of the processing units in the agent and how these units interact, cooperate and synchronize with each other. With the advances in semiconductor technology which allow multiple processing units to be implemented on the same silicon chip, a parallel BDI agent will be an effective way to enable it to perform in a dynamically changing environment when the arrival rate of events is high. We illustrate the working of a parallel agent under the general framework with an agent simulating the behaviour of a vessel captain navigating in sea. Then the performance of a parallel agent is evaluated against several versions of sequential agents. The issue of how much parallelism and how to configure a parallel agent based on the general framework are studied by experiments with different configurations of the parallel agent.

Argumentation and the Dynamics of Warranted Beliefs in Changing Environments

One of the most difficult problems in Multi-Agent Systems (MAS) involves representing the knowledge and beliefs of an agent which performs its tasks in a dynamic environment. New perceptions modify this agent's current knowledge about the world, and consequently its beliefs about it also change. Such a revision and update process should be performed efficiently by the agent, particularly in the context of real-time constraints. In the last decade argumentation has evolved as a successful approach to formalize defeasible, commonsense reasoning, gaining wide acceptance in the MAS community by providing tools for designing and implementing features, which characterize reasoning capabilities in rational agents. In this paper we present a new argument-based formalism specifically designed for representing knowledge and beliefs of agents in dynamic environments, called Observation-based Defeasible Logic Programming (ODeLP). A simple but effective perception mechanism allows an ODeLP-based agent to model new incoming perceptions, and modify the agent's knowledge about the world accordingly. In addition, in order to improve the reactive capabilities of ODeLP-based agents, the process of computing beliefs in a changing environment is made computationally attractive by integrating a database with the agent's program, providing pre-compiled information about previous inferences. We present algorithms for managing dialectical databases as well as examples of their use in the context of real-world problems.

A New HTN Planning Framework for Agents in Dynamic Environments

In a dynamic environment, even if an agent makes a plan to obtain a goal, the environment might change while the agent is executing the plan. In that case, the plan, which was initially valid when it was made, might later become invalid. Furthermore, in the process of replanning, it is necessary to take into account the side effects of actions already executed. Nowadays, HTN planning is becoming popular among agent researchers because its task decomposition algorithm is efficient and suitable for joint planning in multi-agent systems. However, the dynamic replanning algorithm of HTN planning has not yet been established. In order to solve this problem, we have previously presented an agent life cycle that integrates HTN planning, action execution, knowledge updates, and plan modification. In that agent life cycle, the plans are always kept valid according to the most recent knowledge and situation. However, when one plan does not work and the agent starts using an alternative plan, it sometimes undoes some actions that do not have to be undone. The source of this problem lies in the need to have total-order plans. In other words, all the actions have to be sequentially ordered even when the order is not important in some parts of the plan. This paper extends our previous agent life cycle and presents new ways to avoid unnecessary action cancellation. This enables the agent to handle partial-order plans and to make use of two kinds of undoing actions: sequential undoing actions and concurrent undoing actions.

Evaluating new options in the context of existing plans

This paper contributes to the foundations of a theory of rational choice for artificial agents in dynamic environments. Our work is developed within a theoretical framework, originally due to Bratman, that models resource-bounded agents as operating against the background of some current set of intentions, which helps to frame their subsequent reasoning. In contrast to the standard theory of rational choice, where options are evaluated in isolation, we therefore provide an analysis of situations in which the options presented to an agent are evaluated against a background context provided by the agent's current plans—commitments to future activities, which may themselves be only partially specified. The interactions between the new options and the background context can complicate the task of evaluating the option, rendering it either more or less desirable in context than it would have been in isolation.

Cooperative behavior of various agents in dynamic environment

The multi-agent model is a model in which agents with limited ability cooperate each other to accomplish a goal. In this paper, we introduce a multi-agent model in which agents are created to imitate real ants. There are two different type of agents, each type of which has a particular task. We designed agents to communicate each other by using pheromone considering noise. On this model, we observed cooperative behavior of agents and evaluated conditions to make an order of behavior in the model.

Teleo-reactive programs for agent control

A formalism is presented for computing and organizing actions for autonomous agents in dynamic environments. We introduce the notion of teleo-reactive (T-R) programs whose execution entails the con...

Teleo-Reactive Programs for Agent Control

A formalism is presented for computing and organizing actions for autonomous agents in dynamic environments. We introduce the notion of teleo-reactive (T-R) programs whose execution entails the construction of circuitry for the continuous computation of the parameters and conditions on which agent action is based. In addition to continuous feedback, T-R programs support parameter binding and recursion. A primary difference between T-R programs and many other circuit-based systems is that the circuitry of T-R programs is more compact; it is constructed at run time and thus does not have to anticipate all the contingencies that might arise over all possible runs. In addition, T-R programs are intuitive and easy to write and are written in a form that is compatible with automatic planning and learning methods. We briefly describe some experimental applications of T-R programs in the control of simulated and actual mobile robots.