Networked Multi-agent Systems Research Articles

Min–max consensus of multi-agent systems in random networks

This paper focuses on the study of the min–max consensus problem for multi-agent systems in random networks. A min–max consensus algorithm is proposed, which incorporates the weighted average of the agent’s state, the maximum value of its neighbors’ states, and the minimum value of its neighbors’ states. The interaction network is composed of a directed random network where edges exist with probability and are independent of each other. Then, the convergence of the min–max consensus algorithm is investigated utilizing max-plus algebra, min-plus algebra, and stochastic analysis theory. Sufficient and necessary conditions are given to ensure that the multi-agent systems achieve min–max consensus almost surely and in mean-square, respectively. Finally, some numerical simulations are conducted to confirm the effectiveness of the proposed consensus algorithm.

Optimal bipartite consensus control for heterogeneous unknown multi-agent systems via reinforcement learning

This study focuses on addressing optimal bipartite consensus control (OBCC) problems in heterogeneous multi-agent systems (MASs) without relying on the agents' dynamics. Motivated by the need for model-free and optimal consensus control in complex MASs, a novel distributed scheme utilizing reinforcement learning (RL) is proposed to overcome these challenges. The MAS network is randomly partitioned into sub-networks where agents collaborate within each subgroup to attain tracking control and ensure convergence of positions and speeds to a common value. However, agents from distinct subgroups compete to achieve diverse tracking objectives. Furthermore, the heterogeneous MASs considered have unknown first and second-order dynamics, adding to the complexity of the problem. To address the OBCC issue, the policy iteration (PI) algorithm is used to acquire solutions for discrete-time Hamilton-Jacobi-Bellman (HJB) equations while implementing a data-driven actor-critic neural network (ACNN) framework. Ultimately, the accuracy of our proposed approach is confirmed through the presentation of numerical simulations.

Attack-Resilient Distributed Nash Equilibrium Seeking of Uncertain Multiagent Systems Over Unreliable Communication Networks.

This article investigates the distributed Nash equilibrium (NE) seeking problem of uncertain multiagent systems in unreliable communication networks. In this problem, the action of each agent is subject to a class of nonlinear systems with uncertain dynamics, and the communication network among agents will be affected by the nonperiodic denial of service (DoS) attacks. Note that, in this insecure network environment, the existence of DoS attacks will directly destroy the connectivity of the network, which leads to performance degradation or even failure of the most existing distributed NE seeking algorithms. To address this problem, we propose a two-stage distributed NE seeking strategy, including the attack-resilient distributed NE estimator and the neuroadaptive tracking controller. The estimator based on the projection subgradient method and the consensus protocol can converge exponentially to virtual NE against DoS attacks. Then, the neuroadaptive tracking controller is designed for uncertain multiagent systems with the output of the estimator as the reference signal such that the actual action of all agents can reach NE. Based on the Lyapunov stability theory and improved average dwell time automaton, the stability of the estimator and the controller is proven, and all signals in the closed-loop system are uniformly bounded. Numerical examples are presented to verify the effectiveness of the proposed strategy.

Neural Network-Based Distributed Consensus Tracking Control for Nonlinear Multi-Agent Systems with Mismatched and Matched Disturbances

In practice, disturbances, including model uncertainties and unknown external disturbances, are always widely present and have a significant impact on the cooperative control performance of a networked multi-agent system. In this work, the distributed consensus tracking control problem for a class of multi-agent systems subject to matched and mismatched uncertainties is addressed. In particular, the dynamics of the leader agent are modeled with uncertain terms, i.e., the leader’s higher-order information, such as velocity and acceleration, is unknown to all followers. To solve this problem, a robust consensus tracking control scheme that combines a neural network-based distributed observer, a barrier function-based disturbance observer, and a tracking controller based on the back-stepping method was developed in this study. Firstly, a neural network-based distributed observer is designed, which is able to achieve effective estimation of leader information by all followers. Secondly, a tracking controller was designed utilizing the back-stepping technique, and the boundedness of the closed-loop error system was proved using the Lyapunov-like theorem, which enables the followers to effectively track the leader’s trajectory. Meanwhile, a barrier function-based disturbance observer is proposed, which achieves the effective estimation of matched and mismatched uncertainties of followers. Finally, the effectiveness of the robust consensus tracking control method designed in this study was verified through numerical simulations.

Distributed entropy-regularized multi-agent reinforcement learning with policy consensus

Sample efficiency is a limiting factor for existing distributed multi-agent reinforcement learning (MARL) algorithms over networked multi-agent systems. In this paper, the sample efficiency problem is tackled by formally incorporating the entropy regularization into the distributed MARL algorithm design. Firstly, a new entropy-regularized MARL problem is formulated under the model of networked multi-agent Markov decision processes with observation-based policies and homogeneous agents, where the policy parameter sharing among the agents provably preserves the optimality. Secondly, an on-policy distributed actor–critic algorithm is proposed, where each agent shares its parameters of both the critic and actor for consensus update. Then, the convergence analysis of the proposed algorithm is provided based on the stochastic approximation theory under the assumption of linear function approximation of the critic. Furthermore, a practical off-policy version of the proposed algorithm is developed which possesses scalability, data efficiency and learning stability. Finally, the proposed distributed algorithm is compared against the solid baselines including two classic centralized training algorithms in the multi-agent particle environment, whose learning performance is empirically demonstrated through extensive simulation experiments.

Predictive cloud control for nonlinear multiagent systems based on extended state observer and quantizers

AbstractIn this article, predictive cloud control is investigated for nonlinear networked multiagent system (NMAS) with quantizers via extended state observer (ESO). Considering the network‐induced delays in NMAS, the cloud control scheme is designed. Then, arbitrary region quantizers are used to decrease the quantity of data packets. Moreover, the ESO that can estimate the state and uncertainty of nonlinear NMAS is used to obtain the prediction of data packets. Input‐to‐state stability (ISS) can guarantee the validity of ESO. Sufficient conditions that can achieve stability and consistency for NMAS are given by the zoom strategy. The significance of the scheme is verified by simulation results.

Secure Predictive Coordinated Control of High-Order Fully Actuated Networked Multiagent Systems Under Random DoS Attacks.

This research addresses a coordinated control problem for high-order fully actuated networked multiagent systems (HOFANMASs) under random denial-of-service (DoS) attacks. A type of Bernoulli processes is exploited to denote the successful rate of launching random DoS attacks happened to the forward and feedback channels. When acting these attacks successfully, random data losses and disorders are caused in the forward and feedback channels. A high-order fully actuated (HOFA) secure predictive coordinated control scheme is provided to achieve the security coordination. In this scheme, a dynamic model of networked multiagent system is established with the help of a HOFA system model, which is called the HOFANMAS. Then, a prediction model in an incremental HOFA (IHOFA) form is developed by means of a Diophantine equation, which aims at constructing the multistep ahead output predictions for the optimization of coordinated control performance and the compensation of random data losses and disorders. Furthermore, a necessary and sufficient condition is proposed to analyze the consensus and stability of closed-loop HOFANMASs. The effectiveness and superiority of HOFA secure predictive control scheme can be demonstrated via simulated and experimental results of formation control for three air-bearing spacecraft (ABS) simulators.

Max-consensus of multi-agent systems in random networks

This paper considers max-consensus of a discrete-time multi-agent system (MAS) in directed random networks. Interactions among agents in the MAS are probabilistic and independent with each other. By using max-plus algebra and random theory, a sufficient and necessary condition is given for achieving max-consensus of the MAS. Moreover, we demonstrate that the max-consensus in four probabilistic senses (almost surely, in probability, expectation and mean square) is equivalent when expected graph is strongly connected. This ensures that max-consensus can be achieved in multi-agent systems even if random failures occur in the communication network, which is of practical importance in the fields of wireless sensor networks and distributed computing. A simulation example is presented to illustrate the effectiveness of theoretical results.

Quantized consensus of finite-level multi-agent networked systems

<p>In this paper, analysis of a quantized consensus finite-level multi-agent networked system is considered. Logarithmic quantizer is utilized for the transmission of the state of the networked agents. Two protocols employed in the investigation are, the consensus protocol which is determined from the outputs and states of a set of encoder-decoder pair, and the convergence rate protocol that is precisely characterized using a dynamic scaling factor. The asymptotic consensus can be reached through information exchange among neighbour agents. Validity of the protocols for the connected network system is established through careful selection of parameters of the quantizer. A four-agent networked system is used to illustrate the implementation of the protocols. Obtained result showed that information exchange in an undirected networked system is achievable through a communication channel equipped with encoder and decoder pairs. More so, the desired asymptotic convergence is achievable through careful choice of the parameters of the quantizer.</p>

On applications of quasi-Abelian Cayley graphs to Denial-of-Service protection

This paper addresses the problem of designing secure control for networked multi-agent systems (MASs) under Denial-of-Service (DoS) attacks. We propose a constructive design method based on the interaction topology. The MAS with a non-attack communication topology, modeled by quasi-Abelian Cayley graphs subject to DoS attacks, can be represented as a switched system. Using switching theory, we provide easily applicable sufficient conditions for the networked MAS to remain asymptotically stable despite DoS attacks. Our results are applicable to both continuoustime and discrete-time systems, as well as to discrete-time systems with variable steps or systems that combine discrete and continuous times.

On applications of quasi-Abelian Cayley graphs to Denial-of-Service protection

This paper addresses the problem of designing secure control for networked multi-agent systems (MASs) under Denial-of-Service (DoS) attacks. We propose a constructive design method based on the interaction topology. The MAS with a non-attack communication topology, modeled by quasi-Abelian Cayley graphs subject to DoS attacks, can be represented as a switched system. Using switching theory, we provide easily applicable sufficient conditions for the networked MAS to remain asymptotically stable despite DoS attacks. Our results are applicable to both continuoustime and discrete-time systems, as well as to discrete-time systems with variable steps or systems that combine discrete and continuous times.

Adaptive saturated two-bit-triggered bipartite consensus control for networked MASs with periodic disturbances: a low-computation method

Abstract This paper investigates the bipartite tracking control problem for a family of networked multi-agent systems with periodic disturbances as well as input saturation. A low-computation two-bit-triggered adaptive control strategy is proposed to achieve precise trajectory tracking and maintain the boundedness of the closed-loop signals. Compared with the existing results, first, this paper considers the problems for the coexistence of cooperation and competition in multi-agent systems, which represents a more common situation; secondly, the explosion of complexity issue is avoided without introducing any auxiliary filters, making our result more applicable and less complex; thirdly, a function approximator incorporating Fourier series expansion and a radial basis function neural network is utilized to model time-varying periodic disturbance functions and lastly, unlike traditional event-triggered control, the issue of controlling signal transmission bits is further explored to conserve system transmission resources. The result from a comparative simulation illustrates the advantages of the proposed method.

Dynamic Event-Triggered Scaled Consensus of Multi-Agent Systems in Reliable and Unreliable Networks

Dynamic Event-Triggered Scaled Consensus of Multi-Agent Systems in Reliable and Unreliable Networks

Observer-based HOFA predictive cooperative control for networked multi-agent systems under time-variant communication constraints

This research focuses on a cooperative control problem for networked multi-agent systems (NMASs) under time-variant communication constraints (containing time-variant communication delays and time-variant data losses) in the forward and feedback channels. From the perspective of high-order fully actuated (HOFA) system theory, a HOFA system model is adopted to describe the NMAS, which is called the networked HOFA multi-agent system (NHOFAMAS). Because of complicated working scenarios over the network, the states of NMASs are immeasurable and the communication constraints are always present, such that an observer-based HOFA predictive control (OB-HOFAPC) method is designed to implement the cooperative control when existing the immeasurable states and time-variant communication constraints. In this method, a HOFA observer is established to estimate the immeasurable states for constructing a consensus control protocol. Then, an incremental prediction model (IPM) in a HOFA form is developed via a Diophantine equation to take the place of a reduced-order prediction model. Through this IPM, multi-step output ahead predictions are derived to optimize the cooperative control performance and compensate for time-variant communication constraints in real-time. The depth discussion gives a sufficient and necessary criterion to analyze the simultaneous consensus and stability for closed-loop NHOFAMASs. The capability and advantage of OB-HOFAPC method are illustrated via numerical simulation and experimental verification on a cooperative flying-around task of three air-bearing spacecraft simulators.

Controllability of networked multiagent systems based on linearized Turing’s model

Turing’s model has been widely used to explain how simple, uniform structures can give rise to complex, patterned structures during the development of organisms. However, it is very hard to establish rigorous theoretical results for the dynamic evolution behavior of Turing’s model since it is described by nonlinear partial differential equations. We focus on controllability of Turing’s model by linearization and spatial discretization. This linearized model is a networked system whose agents are second order linear systems and these agents interact with each other by Laplacian dynamics on a graph. A control signal can be added to agents of choice. Under mild conditions on the parameters of the linearized Turing’s model, we prove the equivalence between controllability of the linearized Turing’s model and controllability of a Laplace dynamic system with agents of first order dynamics. When the graph is a grid graph or a cylinder grid graph, we then give precisely the minimal number of control nodes and a corresponding control node set such that the Laplace dynamic systems on these graphs with agents of first order dynamics are controllable.

Scaled Consensus of Multi-Agent Systems in the Cooperative-Competitive Networks

Scaled Consensus of Multi-Agent Systems in the Cooperative-Competitive Networks

Consensus of Networked Multi-Agent Systems Operating Over Multiple Communication Channels

Consensus of Networked Multi-Agent Systems Operating Over Multiple Communication Channels

Coalitional Control Strategy for a Heterogeneous Platoon Application

In this work, we propose a coalitional control solution for a multi-agent networked system. The idea is to use a flexible communication topology, in which the communication links are enabled or disabled between sub-systems, depending on their shared task. The control methodology is designed in an optimal control framework, by computing an optimal state feedback gain matrix corresponding to each communication topology. The proposed control algorithm was validated in simulation, and tested in real-time experiments using a heterogeneous vehicle platoon consisting of four mobile robots. Both simulation and experimental results illustrate the efficacy of our proposed methodology.

Global precise consensus tracking control for uncertain multiagent systems in cooperation-competition networks

The multiagent systems (MASs) play a crucial role in practical applications, and the global precise control of MASs under cooperative-competitive networks presents significant challenges. This paper addresses the global precise consensus tracking control problem for uncertain MASs in cooperation-competition networks. A novel framework of distributed event-based prescribed-time observer (EPTO) is proposed for the followers, which allows to accurately estimate the leader's state within a prescribed time interval. Moreover, the design of event-triggered conditions for each EPTO reduces the communication load between neighboring agents by avoiding the continuous transmission of neighbor states. Using the observed values after triggering within the backstepping method framework, a global bipartite control scheme is then developed to expedite the convergence time, enhance the convergence performance, and perform precise tracking control for high-order uncertain MASs. The distinctive feature of the proposed global bipartite control scheme lies in the design of a novel controller, which improves the semi-global control result of the existing schemes. Finally, two simulation examples are presented to demonstrate the effectiveness of the EPTO-based bipartite control scheme.

Coordinated predictive control of networked multiagent systems via distributed cloud computing with time-varying transmission delays

This article examines the complex issue of coordinating networked multi-agent control systems through distributed cloud computing, with a specific focus on compensating for time-varying transmission delays using a predictive control scheme. Unlike the general approach, which treats the time-varying delays as a constant at their maximum value, this paper aims to minimize transmission delays as much as possible. This objective leads to a consensus problem characterized by a time-varying upper-triangular topology. In this study, stability criteria are established to ensure the stability of the time-varying error systems, and a simultaneous similar method is proposed to facilitate feedback designation. The effectiveness of the devised scheme is demonstrated through illustrative examples.