Problem Of Multi-agent Systems Research Articles

Razumikhin and Krasovskii approaches for safe stabilization

This paper studies the stabilization and safety problems of nonlinear time-delay systems. Following both Razumikhin and Krasovskii approaches, we propose novel control Lyapunov functions/functionals for the stabilization problem and novel control barrier functions/functionals for the safety problem. The proposed control Lyapunov and barrier functions/functionals extend the existing ones from the delay-free case to the time-delay case, and allow for designing the stabilizing and safety controllers in closed-form. Since analytical solutions to time-delay optimal control problems are hard to be achieved, a sliding mode control based approach is developed to merge the proposed control Lyapunov and barrier functions/functionals. Based on the sliding surface functional, a feedback control law is established to investigate the stabilization and safety objectives simultaneously. In particular, the properties of the sliding surface functional are analyzed, and further how to construct the sliding surface functional is discussed. Finally, the proposed approaches are illustrated via two numerical examples from the connected cruise control problem of automotive systems and the synchronization problem of multi-agent systems.

Sampled-Data Dynamic Output Feedback Consensus Control of Multi-Agent Systems

This paper investigates the consensus problem of multi-agent systems (MAS) with sampled-data dynamic output feedback (DOF) control. A distributed DOF control law is designed by utilizing sampled relative output information between neighboring agents, and the sampling period can be asynchronous. For the purpose of consensus analysis and appropriately choosing sampling period, a closed-loop system model with hybrid dynamics is constructed, which incorporates flow/jump dynamics and the sampling period. Based on this model, Lyapunov-based analysis is presented and stability conditions are formally developed. Compared with the existing works, observer design is not required and only sampled relative output information is used, and moreover, the maximum allowable sampling period (MASP) can be explicitly computed. Finally, illustrative examples are provided to demonstrate effectiveness of the proposed DOF control method.

Resilient Consensus for Multi-Agent Systems Under Adversarial Spreading Processes

This paper addresses novel consensus problems for multi-agent systems operating in an unreliable environment where adversaries are spreading. The dynamics of the adversarial spreading processes follows the susceptible-infected-recovered (SIR) model, where the infection induces faulty behaviors in the agents and affects their state values. Such a problem setting serves as a model of opinion dynamics in social networks where consensus is to be formed at the time of pandemic and infected individuals may deviate from their true opinions. To ensure resilient consensus among the noninfectious agents, the difficulty is that the number of infectious agents changes over time. We assume that a local policy maker announces the local level of infection in real-time, which can be adopted by the agent for its preventative measures. It is demonstrated that this problem can be formulated as resilient consensus in the presence of the socalled mobile malicious models, where the mean subsequence reduced (MSR) algorithms are known to be effective. We characterize sufficient conditions on the network structures for different policies regarding the announced infection levels and the strength of the epidemic. Numerical simulations are carried out for random graphs to verify the effectiveness of our approach.

Distributed Optimization Algorithms for MASs With Network Attacks: From Continuous-Time to Event-Triggered Communication

In this paper, the distributed optimization problem for multi-agent systems under the existence of cyber attacks is researched. Each agent has a private local cost function which is assumed to be continuously differentiable and strongly convex with the Lipschitz continuous gradient. The global objective function is the sum of these local cost functions, and the communication topologies among agents are balanced and directed. Unlike most multi-agent systems in the existing literature, the cyber attacks in networks, which may result in the failure of information transmission, are considered. Two novel distributed optimization algorithms with the continuous-time communication are studied for such an optimization problem. In addition, in order to reduce computation and communication consumption efficiently, we further enhance these algorithms with an event-triggered communication mechanism. Under some assumptions and restrictions on attacks, sufficient yet efficient conditions for the convergence of the proposed algorithms are derived. Furthermore, the Zeno-behavior is shown to be excluded for the event-triggered distributed optimization algorithms. Finally, some numerical simulation examples are provided to illustrate and validate the efficiency of the proposed optimization algorithms.

Optimal Leader-Following Consensus Control of Multi-Agent Systems: A Neural Network Based Graphical Game Approach

In this article, the optimal leader-following consensus control problem of multi-agent systems is solved using a novel neural network-based (NN-based) integrated heuristic dynamic programming (IHDP) algorithm under a distributed learning framework. The optimal controller is designed to find a Nash equilibrium of the multi-agent differential graphical game. According to the principle of optimality, this problem can be transformed into solving a set of coupled Hamilton-Jacobi-Bellman (HJB) equations, which are difficult to solve directly because of the nonlinearity and strong coupling of the equations. The proposed algorithm combines policy iteration (PI) and value iteration (VI), and the Nash equilibrium of performance index among agents is achieved distributedly, i.e., all agents update their control inputs only use local information. The convergence of the algorithm is rigorously proved. Neural network (NN) approximation implementation is given for the unknown structures of local value functions of agents. The effectiveness of the algorithm is verified by numerical simulation.

Obstacle avoidance flocking motion in multi-agent systems with limited sensing radius and heterogeneous input constraints

This paper addresses the leaderless and leader-followers flocking motion problems of multi-agent systems in a workspace with obstacles. We consider inertial agents with equally limited communication radius and heterogeneous input constraints. Using the notion of proximity graphs, we provide distributed controllers to induce the desired flocking behaviour. Unlike previous results, our controller designs do not require predicted states or a target determination scheme and allow edge deletions. Also, by taking advantage of the group's heterogeneity, our controllers allow agents with less restrictive input constraints to compensate for the control effort that their less capable neighbours cannot provide. We illustrate the effectiveness of our proposals through numerical simulations.

Robust secure consensus of multiagent systems with DoS attacks

The secure consensus problem of multiagent systems with fixed topology and external disturbances is investigated in this paper. Along with external disturbances, the studied multiagent system is also subjected to denial-of-service (DoS) attacks. First, some graph-based Lyapunov functions are presented for the robust consensus analysis. Second, the duration and attack frequency are introduced to quantitatively analyze the DoS attacks’ effects on the consensus. Third, the gain of the controller is acquired by solving an algebraic Riccati equation (ARE), which can always be guaranteed compared with solving linear matrix inequalities (LMIs). Finally, a numerical simulation of a microgrid test system is provided to demonstrate the effectiveness of the proposed strategy.

Optimization Over Time-Varying Networks and Unbounded Information Delays

Solving optimization problems in multiagent systems involves information exchange between agents. The obtained solutions should be robust to information delays and errors that arise from an unreliable wireless network, which typically connects the multiagent system. In today’s large-scale dynamic Internet of Things style multiagent scenarios, the network topology changes and evolves over time. In this article, we present a simple distributed gradient-based optimization framework and an associated algorithm. Convergence to a minimum of a given objective is shown under mild conditions on the network topology and objective. A key feature of our approach is that we merely assume that the messages sent reach the intended receiver, possibly delayed, with some positive probability. To the best of authors’ knowledge, ours is the first analysis under such weak general network conditions. We also discuss in detail the verifiability of the assumptions involved. This article also makes a technical contribution in terms of the allowed class of objective functions. Specifically, we present an analysis wherein the objective function is such that its sample-gradient is merely locally Lipschitz continuous. The theory developed herein is supported by empirical results.

Cooperative Output Regulation Problem for Multiagent System Under Directed Topology Using Dynamic Triggering Strategy

This brief focuses on cooperative output regulation (COR) problem for linear multiagent systems (MASs) under the directed topology. The fully distributed event-triggered control protocol is designed to solve COR problem. In order to reduce the communication frequency, the dynamic event-triggered mechanism is proposed by introducing positive internal variable. One main advantage is that the method inherits the advantages of event-triggered control and adaptive control to deal with the COR problem, and does not depend on global information. At last, a numerical example is given in the supplementary material to verify the validity of the obtained results.

Better Approximation for Distributed Weighted Vertex Cover via Game-Theoretic Learning

Toward better approximation for the minimum-weighted vertex cover (MWVC) problem in multiagent systems, we present a distributed algorithm from the perspective of learning in games. For self-organized coordination and optimization, we see each vertex as a potential game player who makes decisions using local information of its own and the immediate neighbors. The resulting Nash equilibrium is classified into two categories, i.e., the inferior Nash equilibrium (INE) and the dominant Nash equilibrium (DNE). We show that the optimal solution must be a DNE. To achieve better approximation ratios, local rules of perturbation and weighted memory are designed, with the former destroying the stability of an INE and the latter facilitating the refinement of a DNE. By showing the existence of an improvement path from any INE to a DNE, we prove that when the memory length is larger than 1, our algorithm converges in finite time to DNEs, which could not be improved by exchanging the action of a selected node with all its unselected neighbors. Moreover, additional freedom for solution efficiency refinement is provided by increasing the memory length. Finally, intensive comparison experiments demonstrate the superiority of the presented methodology to the state of the art, both in solution efficiency and computation speed.

Optimal Consensus via OCPI Regulation for Unknown Pure-Feedback Agents With Disturbances and State Delays

In this article, we present a novel methodology to address the optimal output consensus problem for multiagent systems. We propose a distributed variable transformation that recasts the aforementioned problem into a simpler regulation problem of the new variables. The transformation requires only relative output measurements and is independent of any underlying agent dynamics and therefore applicable to a general class of systems. Classical control techniques, which take into account the agent dynamics and only need the relative outputs to be shared between neighbors, can then be employed to regulate the new variables. We utilize the methodology to address, for the first time, the problem of optimal output consensus for unknown pure-feedback agents with disturbances and state delays. Simulation studies on a group of continuous stirred tank reactors illustrate the validity of the theoretical analysis.

Sampled-data-based consensus of multi-agent systems with multiplicative noise and time-delays

The sampled-data-based consensus control problem of multi-agent systems (MASs) with multiplicative noise and time-delays is studied in this paper. The MASs are given in a continuous-time setting while information exchange occurs only at the sampling instants. Therefore, the traditional methods for continuous-time or discrete-time models are not applicable. To resolve the problem, a discretization-reconnection method is developed to investigate the consensus control of first-order and second-order MASs, respectively. Some sufficient conditions, explicitly related to the control gains and the sampling step size, are established for both mean-square (m.s.) and almost sure (a.s.) consensus. Under these conditions, consensus can be achieved by designing appropriate control gains and the sampling step size for MASs with given noise intensity and time-delays. Numerical simulations verify the effectiveness of the theoretical results.

Nonlinear finite-time and fixed-time bipartite consensus tracking of multi-agent systems

ABSTRACT This paper deals with the finite-time and fixed-time bipartite consensus tracking (FFBCT) problems for multi-agent systems (MASs) , in which both cooperative and competition exist. Using Lyapunov stability methods, distributed nonlinear protocols are provided to ensure all agents reach agreement. Its absolute value is the same but it might be different in direction. Besides, the second protocol can make the system reach bipartite consensus in fixed time regardless of the initial state of agents. Moreover, boundaries for settling times are estimated. Some numerical examples are finally presented to prove the validity of our designs.

A new performance bound for submodular maximization problems and its application to multi-agent optimal coverage problems

Several important problems in multi-agent systems, machine learning, data mining, scheduling and others, may be formulated as set function maximization problems subject to cardinality constraints. In such problems, the set (objective) functions of interest often have monotonicity and submodularity properties. Hence, the class of monotone submodular set function maximization problems has been widely studied in the literature. Owing to its challenging nature, almost all existing solutions for this class of problems are based on greedy algorithms. A seminal work on this topic has exploited the submodularity property to prove a (1-1/e) performance bound for such greedy solutions. More recent literature on this topic has been focused on exploiting different curvature properties to establish improved (tighter) performance bounds. However, such improvements come at the cost of enforcing additional assumptions and increasing computational complexity while facing significant inherent limitations. In this paper, first, a brief review of existing performance bounds is provided. Then, a new performance bound that does not require any additional assumptions and is both practical and computationally inexpensive is proposed. In particular, this new performance bound is established based on a series of upper bounds derived for the objective function that can be computed in parallel with the execution of the greedy algorithm. Finally, to highlight the effectiveness of the proposed performance bound, extensive numerical results obtained from a well-known class of multi-agent coverage problems are provided.

Consensus of linear multi-agent systems by distributed event-triggered strategy with designable minimum inter-event time

This paper investigates the consensus problem of multi-agent systems under the event-triggered control with designable minimum inter-event time, in which each agent has an identical linear dynamic model. The goal is to devise a novel control method which can not only reduce actuator consumption but also guarantee the state consensus of the controlled systems. Two event-triggered mechanisms are considered, namely, the static event-triggered mechanism and the dynamic event-triggered mechanism. To this end, a novel distributed static event-triggered control algorithm is presented at first. Then the distributed dynamic event-triggered control algorithm is developed. The major superiority of the presented distributed static and dynamic event-triggered control algorithms is that the controller of each agent is only triggered at its own event time, which effectively reduces the update frequency of the controller in practice. Further, the minimum inter-event time is adjustable by the parameter of the parametric Lyapunov equation, which makes it easier to find a trade-off between the inter-event times and the control performance of the controlled system. Subsequently, it is proved that the consensus of the multi-agent systems is reached asymptotically and non-occurrence of the Zeno phenomenon is proved. The superiority of the designed algorithms is testified by the simulation.

Learning to Utilize Curiosity: A New Approach of Automatic Curriculum Learning for Deep RL

In recent years, reinforcement learning algorithms based on automatic curriculum learning have been increasingly applied to multi-agent system problems. However, in the sparse reward environment, the reinforcement learning agents get almost no feedback from the environment during the whole training process, which leads to a decrease in the convergence speed and learning efficiency of the curriculum reinforcement learning algorithm. Based on the automatic curriculum learning algorithm, this paper proposes a curriculum reinforcement learning method based on the curiosity model (CMCL). The method divides the curriculum sorting criteria into temporal-difference error and curiosity reward, uses the K-fold cross validation method to evaluate the difficulty priority of task samples, uses the Intrinsic Curiosity Module (ICM) to evaluate the curiosity priority of the task samples, and uses the curriculum factor to adjust the learning probability of the task samples. This study compares the CMCL algorithm with other baseline algorithms in cooperative-competitive environments, and the experimental simulation results show that the CMCL method can improve the training performance and robustness of multi-agent deep reinforcement learning algorithms.

Distributed algorithm design for constrained resource allocation problems with high-order multi-agent systems

This paper studies distributed resource allocation problems of multi-agent systems, where the decisions of agents are subject to inequality network resource constraints and local inequality constraints. Compared with well-known resource allocation problems, our problem considers the high-order dynamics of agents. Without involving the control of high-order dynamics, existing distributed resource allocation algorithms cannot deal with our problem. Meanwhile, the high-order dynamics together with the inequality constraints makes it difficult to design and analyze distributed resource allocation algorithms, because the outputs of agents cannot be decided by their inputs directly and the optimal decisions of agents must satisfy the inequality constraints. In order to control the high-order agents to accomplish the resource allocation tasks autonomously, a distributed algorithm is designed by state feedback, gradient descent and primal–dual methods. Moreover, the convergence of the algorithm is analyzed by convex analysis and Lyapunov stability theory. With the algorithm, the high-order agents converge to the optimal allocation. Finally, numerical simulations verify the effectiveness of the algorithm.

Sample-based Dynamic Event-triggered Algorithm for Optimization Problem of Multi-agent Systems

Sample-based Dynamic Event-triggered Algorithm for Optimization Problem of Multi-agent Systems

Compartmental observability approach for the optimal transparency problem in multi-agent systems

This paper considers a network of agents, where each agent is assumed to take actions optimally with respect to a predefined payoff function involving the latest actions of the agent’s neighbors. Neighborhood relationships stem from payoff functions rather than actual communication channels between the agents. A principal is tasked to optimize the network’s performance by controlling the information available to each agent with regard to other agents’ latest actions. The information control by the principal is done via a compartmental observability approach, which comprises a static partitioning of agents into blocks and making the mean of agents’ latest actions within each block publicly available. While the problem setup is very general in terms of the payoff functions and the network’s performance metric, the paper has a narrower, yet still fairly general, focus to display essential characteristics of the problem and to show how it can be addressed in practice. In particular, the performance metric is assumed to be a function of the steady-state behavior of the agents. After conducting a comprehensive steady-state analysis of the network, two efficient heuristic algorithms finding nearly-optimal partitions with respect to various performance metrics are presented and validated via numerical studies.

A new perspective on cooperative control of multi-agent systems through different types of graph Laplacians

In the field of systems and control, many cooperative control problems of multi-agent systems have been actively studied in the past two decades. This article aims to organize extensive existing work on different cooperative control problems into three categories, based on three different types of graph Laplacian matrices involved. A Laplacian matrix is an important representation of graph topology, and depending on the field of its entries, there are three types: ordinary Laplacian (non-negative diagonal entries and non-positive off-diagonal entries), signed Laplacian (arbitrary real entries), and complex Laplacian (arbitrary complex entries). Each type of graph Laplacian is useful in modeling and solving a different set of cooperative control problems. In particular, their algebraic properties are fundamental in characterizing stability and performance of the respective solution algorithms. To our best knowledge, organizing the literature on multi-agent cooperative control through the lens of different graph Laplacians is new.