Group Of Mobile Agents Research Articles

Detection of Hidden Moving Targets by a Group of Mobile Agents with Deep Q-Learning

In this paper, we propose a solution for the problem of searching for multiple targets by a group of mobile agents with sensing errors of the first and the second types. The agents’ goal is to plan the search and follow its trajectories that lead to target detection in minimal time. Relying on real sensors’ properties, we assume that the agents can detect the targets in various directions and distances; however, they are exposed to first- and second-type statistical errors. Furthermore, we assume that the agents in the group have errorless communication with each other. No central station or coordinating agent is assumed to control the search. Thus, the search follows a fully distributed decision-making process, in which each agent plans its path independently based on the information about the targets, which is collected independently or received from the other agents. The suggested solution includes two algorithms: the Distributed Expected Information Gain (DEIG) algorithm, which implements dynamic Voronoi partitioning of the search space and plans the paths by maximizing the expected one-step look-ahead information per region, and the Collective Q-max (CQM) algorithm, which finds the shortest paths of the agents in the group by maximizing the cumulative information about the targets’ locations using deep Q-learning techniques. The developed algorithms are compared against previously developed reactive and learning methods, such as the greedy centralized Expected Information Gain (EIG) method. It is demonstrated that these algorithms, specifically the Collective Q-max algorithm, considerably outperform existing solutions. In particular, the proposed algorithms improve the results by 20% to 100% under different scenarios of noisy environments and sensors’ sensitivity.

Nonlinear Algorithms for Controlling a Group of Mobile Agents on a Segment

A group of mobile agents on a straight line is considered. Agents are understood as numbered points that can change their position. It is assumed that the dynamics of agents is modeled by second-order integrators, with each agent receiving information from one of its left and one of its right neighbors (not necessarily nearest neighbors). It is required to provide a given nonlinear-uniform (uniform with respect to a prescribed nonlinear function) deployment of agents on a straight line segment. It is worth mentioning that, in numerous applications such as optic, acoustics, physiology, information theory, thermodynamics, etc., instead of linear scale, nonlinear ones (for instance, logarithmic) are used. In addition, it should be noted that an important class of formation control problems is synchronization of processes with respect to certain functions of phase coordinates. To solve the stated problem, nonlinear decentralized protocols are proposed. The conditions on the control parameters are determined, under which the agents converge to the required positions. The robustness of the constructed control protocols with respect to communication delay and network topology switching (replacing chosen neighbors by the other ones) is investigated. In this case, it is assumed that information about the magnitude of the delay and about the switching law may be absent. It is shown that for any constant non-negative delay and any admissible law for switching connections, a given deployment of agents is guaranteed. The proofs of the stated statements are based on the application of the Lyapunov direct method and a special form of the decomposition method. Original constructions of Lyapunov functions and Lyapunov—Krasovskii functionals are used. The results of numerical simulation are presented, confirming the obtained theoretical conclusions.

Multi-agent reinforcement learning for long-term network resource allocation through auction: A V2X application

We formulate offloading of computational tasks from a dynamic group of mobile agents (e.g., cars) as decentralized decision making among autonomous agents. We design an interaction mechanism that incentivizes such agents to align private and system goals by balancing between competition and cooperation. In the static case, the mechanism provably has Nash equilibria with optimal resource allocation. In a dynamic environment, this mechanism’s requirement of complete information is impossible to achieve. For such environments, we propose a novel multi-agent online learning algorithm that learns with partial, delayed and noisy state information, thus greatly reducing information need. Our algorithm is also capable of learning from long-term and sparse reward signals with varying delay. Empirical results from the simulation of a V2X application confirm that through learning, agents with the learning algorithm significantly improve both system and individual performance, reducing up to 30% of offloading failure rate, communication overhead and load variation, increasing computation resource utilization and fairness. Results also confirm the algorithm’s good convergence and generalization property in different environments.

Doubly Stochastic Pairwise Interactions for Agreement and Alignment

Random pairwise encounters often occur in large populations, or groups of mobile agents, and various types of local interactions that happen at encounters account for emergent global phenomena. In particular, in the fields of swarm robotics, sociobiology, and social dynamics, several types of local pairwise interactions were proposed and analysed leading to spatial gathering or clustering and agreement in teams of robotic agents coordinated motion, in animal herds, or in human societies. We here propose a very simple stochastic interaction at encounters that leads to agreement or geometric alignment in swarms of simple agents, and analyse the process of converging to consensus. Consider a group of agents whose evolve in time by pairwise interactions: the state of an agent is either a real value (a randomly initialised position within an interval) or a vector that is either unconstrained (e.g. the location of the agent in the plane) or constrained to have unit length (e.g. the direction of the agent's motion). The interactions are doubly stochastic, in the sense that, at discrete time steps, pairs of agents are randomly selected and their new states are independently and uniformly set at random in (local) domains or intervals defined by the states of the interacting pair. We show that such processes lead, in finite expected time (measured by the number of interactions that occurred) to agreement in case of unconstrained states and alignment when the states are unit vectors.

A Model of a Decentralized Cyber-Physical System Resiliently Functioning in a Changing Environment

The article discusses the issues of decentralization and dynamic redistribution of roles in cyber-physical systems (CPS) designed for working in changing environments and especially open spaces, where there are increased risks of module failures and communication loss. In particular, decentralized methods for controlling the CPS behavior and ensuring the redundancy of their components and connections are investigated. A number of requirements for such systems are identified and it is noted what limitations in existing approaches impede the implementation of systems that satisfy these requirements at the physical, network and application layers. For different layers, behavior models are proposed, which provide autonomous role distribution between components. This made it possible to synthesize a structural-parametric model of an autonomous mobile CPS, focused on functioning in open areas and solving applied problems performed through the coordinated interaction of groups of mobile agents. The model takes into account aspects of system stability and its response to destructive influences. The advantages of the proposed models include task decentralization, absence of central, critical nodes and bottlenecks, no line of sight requirements or small distance between devices, ability to work in unexplored environments. The solutions can be applied primarily in the field of business and are suitable for use in industrial plants equipped with mobile robotic devices with cameras, for example, for agricultural tasks, or territorial exploration. The proposed approach makes possible to study the terrain more accurately, including unexplored areas with limited accessibility to humans. Also, the results obtained can be applied during rescue operations.

Coverage control of mobile agents using multi-step broadcast control

AbstractThis paper proposes a novel multi-step broadcast control (MBC) scheme to deploy a group of autonomous mobile agents for accomplishing coverage tasks in a bounded region. Traditional broadcast control (BC) schemes use a one-to-all communication framework to transmit a uniform signal to all agents, making it cost-effective compared with any all-to-all communication-based scheme for a multi-agent system. However, as BC schemes are based on a single-step view of the environment for decision-making, the environment’s varying distribution density is not known immediately to the agents, resulting in suboptimal performance. To overcome this drawback, this paper proposes an MBC scheme, where agents use a predictive multi-step view and are able to detect the varying densities in the environment ahead of time. The local controller output is estimated using a weighted averaging technique which assigns a higher weight to immediate steps; this feature compensates for any decrease in prediction accuracy as the number of steps increases. We demonstrate the effectiveness of the proposed MBC scheme using a coverage task over a region with uneven population density. Compared to existing BC schemes, the proposed MBC scheme shows superior convergence characteristics in task accomplishment and deployment efficiency.

Broadcast Guidance of Multi-Agent Systems

We consider the emergent behavior of a group of mobile agents guided by an exogenous broadcast signal. The agents’ dynamics is modelled by single integrators and they are assumed oblivious to their own position, however they share a common orientation (i.e. they have compasses). The broadcast control, a desired velocity vector, is detected by arbitrary subgroups of agents,that upon receipt of the guidance signal become "ad-hoc" leaders. The control signal and the set of leaders are assumed to be constant over some considerable intervals in time. A system without "ad-hoc" leaders is referred to as autonomous. The autonomous rule of motion is identical for all agents and is a gathering process ensuring a cohesive group. The agents that become leaders upon receipt of the exogenous control add the detected broadcast velocity to the velocity vector dictated by the autonomous rule of motion. This paradigm was considered in conjunction with several models of cohesive dynamics, linear and non-linear, with fixed inter-agent interaction topology, as well as systems with neighborhood based topology determined by the inter-agent distances. The autonomous dynamics of the models considered provides cohesion to the swarm, while, upon detection of a broadcast velocity vector, the leaders guide the group of agents in the direction of the control. For each local cohesion interaction model we analyse the effect of the broadcast velocity and of the set of leaders on the emergent behavior of the system. We show that in all cases considered the swarm moves in the direction of the broadcast velocity signal with speed set by the number of agents receiving the control and in a constellation determined by the model and the subset of "ad-hoc" leaders. All results are illustrated by simulations.

Gaussian Mixture Based Motion Prediction for Cluster Groups of Mobile Agents

This paper presents a Gaussian mixture model based method for estimation of multiple mobile agent trajectories over time following a social force motion model. From an initial observation, groups of agents are formed using a bottom-up agglomerate clustering approach. Each cluster constitutes a component of the Gaussian mixture model, with mean position of the cluster members. An Adaptive Gaussian Sum filter estimates the future trajectories of the agent clusters using the Gaussian mixture model input. The proposed algorithm is capable of reducing the computation time compared to the standard Adaptive Gaussian Sum filter and provides solutions with lower variance due to utilization of agent groups. The efficacy of the proposed method is demonstrated via extensive numerical simulations.

Moving-Target Enclosing Control for Mobile Agents With Collision Avoidance

This article studies the moving-target enclosing control problem for a group of mobile agents, under the assumption that the amplitude of the target's velocity is unknown <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">a priori</i> . The target's velocity is time-varying and can be viewed as a signal generated by an exogenous linear system. The neighbor topology of the agents is determined by their positions relative to the target at each time instant. Based on the relative positions, a distributed observer is proposed for the agents to cooperatively estimate the velocity of the target. Then, a novel distributed control law is proposed based on the obtained estimates. It is shown that under the proposed control law, the agents eventually move along a common circle of the desired radius around the moving target and achieve any desired spaced pattern along the circle. Furthermore, conditions are derived under which the collision between each agent and the target is avoided, and the interagent collision avoidance is also guaranteed by preserving the counterclockwise order of the agents around the target. Finally, a simulation example illustrates the effectiveness of the proposed control law.

A problem of formation control on a line segment under protocols with communication delay

The problem of the equidistant deployment of a group of mobile agents on a line segment is studied. Both cases where agent dynamics is modeled by the first-order integrators and double integrators are considered. It is assumed that each agent receives information from its neighbors not directly, but via an auxiliary agent. The impact of communication delays on the agent deployment is investigated. Conditions are derived ensuring the convergence of agents to the equidistant distribution for any constant nonnegative delays. Efficiency of the obtained results is demonstrated by a numerical simulation.

A sub-modular receding horizon solution for mobile multi-agent persistent monitoring

We consider persistent monitoring of a finite number of inter-connected geographical nodes by a group of heterogeneous mobile agents. We assign to each geographical node a concave and increasing reward function that resets to zero after an agent’s visit. Then, we design the optimal dispatch policy of which nodes to visit at what time and by what agent by finding a policy set that maximizes a utility that is defined as the total reward collected at visit times. We show that this optimization problem is NP-hard and its computational complexity increases exponentially with the number of the agents and the length of the mission horizon. By showing that the utility function is a monotone increasing and submodular set function of agents’ policy, we propose a suboptimal dispatch policy design with a known optimality gap. To reduce the time complexity of constructing the feasible search set and also to induce robustness to changes in the operational factors, we perform our suboptimal policy design in a receding horizon fashion. Then, to compensate for the shortsightedness of the receding horizon approach we add a new term to our utility, which provides a measure of nodal importance beyond the receding horizon. This term gives the policy design an intuition to steer the agents towards the nodes with higher rewards on the patrolling graph. Finally, we discuss how our proposed algorithm can be implemented in a decentralized manner. A simulation study demonstrates our results.

Bounded and inverse optimal formation stabilization of second-order agents

This paper formulates and solves a new problem of both bounded and inverse optimal formation stabilization control for a group of second-order dynamic mobile agents with collision avoidance and limited sensing range. The control design is based on new Lyapunov functions, new non-zero convergence and dominating lemmas, new pairwise collision avoidance functions, and forwarding and inverse optimal control design methods. The proposed formation stabilization control design guarantees no collision between any agents, “almost global” asymptotic stability of desired equilibrium points and instability of undesired equilibrium points, an infinite gain margin, bounded controls by a pre-specified constant, and minimization of a cost function that penalizes both stabilization errors and the control inputs without having to solve a Hamilton–Jacobi–Bellman or Hamilton–Jacobi–Isaacs equation.

A Cooperative Multiagent Probabilistic Framework for Search and Track Missions

In this work a robust and scalable cooperative multi-agent searching and tracking framework is proposed. Specifically, we study the problem of cooperative searching and tracking of multiple moving targets by a group of autonomous mobile agents with limited sensing capabilities. We assume that the actual number of targets present is not known a priori and that target births/deaths can occur anywhere inside the surveillance region thus efficient search strategies are required to detect and track as many targets as possible. To address the aforementioned challenges we recursively compute and propagate in time the searching-and-tracking (SAT) density. Using the SAT-density, we then develop decentralized cooperative look-ahead strategies for efficient searching and tracking of an unknown number of targets inside a bounded surveillance area.

Jointly-Optimized Searching and Tracking with Random Finite Sets

In this paper, we investigate the problem of joint searching and tracking of multiple mobile targets by a group of mobile agents. The targets appear and disappear at random times inside a surveillance region and their positions are random and unknown. The agents have limited sensing range and receive noisy measurements from the targets. A decision and control problem arises, where the mode of operation (i.e., search or track) as well as the mobility control action for each agent, at each time instance, must be determined so that the collective goal of searching and tracking is achieved. We build our approach upon the theory of random finite sets (RFS) and we use Bayesian multi-object stochastic filtering to simultaneously estimate the time-varying number of targets and their states from a sequence of noisy measurements. We formulate the above problem as a non-linear binary program (NLBP) and show that it can be approximated by a genetic algorithm. Finally, to study the effectiveness and performance of the proposed approach we have conducted extensive simulation experiments.

Inverse optimal formation control of mobile agents with collision avoidance driven by Lévy processes

This paper formulates and solves a new problem of formation control for a group of second-order dynamic mobile agents with collision avoidance driven by Lévy processes with unknown characteristics. First, a new result on inverse optimal control design for nonlinear systems driven by Lévy processes without having to solve a Hamilton–Jacobi–Bellman equation or a Hamilton–Jaccobi–Isaacs equation is developed. Second, the newly developed result together with the Lyapunov direct and backstepping methods, and potential functions will be then used to solve the posed inverse optimal formation control problem for the mobile agents with limited sensing ranges.

Cooperative Detection of Multiple Targets by the Group of Mobile Agents

The paper considers the detection of multiple targets by a group of mobile robots that perform under uncertainty. The agents are equipped with sensors with positive and non-negligible probabilities of detecting the targets at different distances. The goal is to define the trajectories of the agents that can lead to the detection of the targets in minimal time. The suggested solution follows the classical Koopman’s approach applied to an occupancy grid, while the decision-making and control schemes are conducted based on information-theoretic criteria. Sensor fusion in each agent and over the agents is implemented using a general Bayesian scheme. The presented procedures follow the expected information gain approach utilizing the “center of view” and the “center of gravity” algorithms. These methods are compared with a simulated learning method. The activity of the procedures is analyzed using numerical simulations.

Distributed Optimal Deployment on a Circle for Cooperative Encirclement of Autonomous Mobile Multi-Agents

A distributed encirclement points deployment scheme for a group of autonomous mobile agents is addressed in this paper. Herein, each agent can measure its own azimuth related to the common target and can at least communicate with its two adjacent neighbors. Given its space-cooperative character, the encirclement points deployment problem is formulated as the coverage control problem on a circle. The measurement range of azimuth sensor is taken into consideration when doing problem formulation, which is closer to the facts in real-world applications. Then, the fully distributed control protocols are put forward based on geometric principle and the convergence is proved strictly with algebraic method. The proposed control protocols can steer the mobile agents to distribute evenly on the circle such that the coverage cost function is minimized, and meanwhile the mobile agents' spatial order on the circle is preserved throughout the systems' evolution. A noteworthy feature of the proposed control protocols is that only the azimuths of a mobile agent and its two adjacent neighbors are needed to calculate the mobile agent's control input, so that the control protocols can be easily implemented in general. Moreover, an adjustable feedback gain is introduced, and it can be employed to improve the convergence rate effectively. Finally, numerical simulations are carried out to verify the effectiveness of the proposed distributed control protocols.

Flocking and topology manipulation based on space partitioning

Network topology plays a critical role in enabling a multi-agent system to adapt to environment changes and achieve desired objectives. This paper presents distributed topology manipulation schemes for a group of mobile agents. The agents have limited heterogeneous communication ranges, and connections among them are directional. The topology is established from the overlapping communication ranges. The admissible space is partitioned into enclosed areas by connectivity among the agents based on their communication ranges. Each agent occupies an enclosed area, and its decision-making manipulates the topology by guiding itself to an adjacent enclosed area. Both independent and coordinated decision-making approaches are provided. A guidance algorithm is designed to drive the vehicles to a flexible formation, in which the robustness of the network topology is enhanced.

Limit-Cycle-Based Design of Formation Control for Mobile Agents

In this article, we study the formation problem for a group of mobile agents in a plane, in which the agents are required to maintain a distribution pattern, as well as to rotate around or remain static relative to a static/moving target. The prescribed distribution pattern is a class of general formations where the distances between neighboring agents or the distances between each agent and the target do not need to be equal. Each agent is modeled as a double integrator and can merely perceive the relative information of the target and its neighbors, and the acceleration of the target. In order to solve the formation problem, a limit-cycle-based controller design is delivered. We divide the overall control objective into two subobjectives, where the first is target circling that each agent keeps its own desired distance to the static/moving target as well as rotating around or remaining static relative to the target as expected, and the second is distribution adjustment that each agent maintains the desired distance to its neighbors. Then, we propose a controller comprised of two parts, where a limit cycle oscillator named a converging part is designed to deal with the first subobjective, while a layout part is introduced to address the second subobjective. One key merit of the controller is that it can be implemented by each agent in its local frame so that only local information is utilized without knowing global information. Theoretical analysis of the convergence to the desired formation, of which the agents are required to be evenly distributed on a circle around the target, is provided for the multiagent system under the proposed controller. Numerical simulations are given to validate the effectiveness of the proposed controller for the cases of general formations, and to show that no collision between agents ever takes place throughout the system's evolution.

Cooperative Periodic Coverage With Collision Avoidance

In this paper, we propose a periodic solution to the problem of persistently covering a finite set of interest points with a group of autonomous mobile agents. These agents visit periodically the points and spend some time carrying out the coverage task, which we call coverage time. Since this periodic persistent coverage problem is NP-hard, we split it into three subproblems to counteract its complexity. In the first place, we plan individual closed paths for the agents to cover all the points. Second, we formulate a quadratically constrained linear program to find the optimal coverage times and actions that satisfy the coverage objective. Finally, we join together the individual plans of the agents in a periodic team plan by obtaining a schedule that guarantees collision avoidance. To this end, we solve a mixed-integer linear program that minimizes the time in which two or more agents move at the same time. Eventually, we apply the proposed solution to an induction hob with mobile inductors for a domestic heating application and show its performance with experiments on a real prototype.