Second-order Discrete-time Multi-agent Systems Research Articles

Data-Driven Optimal Bipartite Consensus Control for Second-Order Multiagent Systems via Policy Gradient Reinforcement Learning.

This article investigates the optimal bipartite consensus control (OBCC) problem for unknown second-order discrete-time multiagent systems (MASs). First, the coopetition network is constructed to describe the cooperative and competitive relationships between agents, and the OBCC problem is proposed by the tracking error and related performance index function. Based on the distributed policy gradient reinforcement learning (RL) theory, a data-driven distributed optimal control strategy is obtained to guarantee the bipartite consensus of all agents' position and velocity states. In addition, the offline data sets ensure the learning efficiency of the system. These data sets are generated by running the system in real time. Besides, the designed algorithm is an asynchronous version, which is essential to solve the challenge caused by the computational ability difference between nodes in MASs. Then, by means of the functional analysis and Lyapunov theory, the stability of the proposed MASs and the convergence of the learning process are analyzed. Furthermore, an actor-critic structure containing two neural networks is used to implement the proposed methods. Finally, a numerical simulation shows the effectiveness and validity of the results.

Differentially Private Consensus for Second-Order Multiagent Systems With Quantized Communication.

This article considers the differentially private consensus problem of discrete-time second-order multiagent systems with partially measurable states and limited communication channel capacity, where only the integer-value information of agents can be transmitted. To reduce the potential risk of state information disclosure in digital communication, a differentially private consensus algorithm via dynamic encoding-decoding is proposed for the second-order multiagent system to make agents achieve mean-square consensus by transmitting quantized integer values with privacy protection. To deal with the uncertainty of the quantizer saturation, the statistical analysis is given for the boundedness of the input of quantizers. It is shown that the expectation of the minimum memory capacity of quantizers is 2 bits. Finally, some simulation results are given to visualize our conclusions.

Matrix-Weighted Consensus of Second-Order Discrete-Time Multiagent Systems.

In this article, we study the matrix-weighted consensus issues for second-order discrete-time multiagent systems on directed network topology. Under the designed matrix-weighted consensus algorithm, based on the eigenvalues of the Laplacian matrix, coupling gains, and discrete interval, we build some consensus conditions for reaching discrete-time consensus and deduce some simplified and straightforward consensus conditions for undirected network topology. Besides, for a given network topology, we theoretically analyze the influence of the coupling gains and discrete intervals on the consensus conditions of the network dynamics. Finally, we offer several simulation examples to validate the obtained results.

Strong reciprocal ratio consensus of modified second-order multi-agent systems with periodically time-varying topologies

The strong reciprocal ratio consensus problem of modified second-order discrete-time multi-agent systems (DTMASs) with periodically time-varying topologies is investigated. The strong reciprocal ratio consensus is a new consensus and covers the strong ratio average-consensus as a special case. Modified second-order DTMAS is a variant of second-order DTMAS. A distributed control protocol is designed to achieve the strong reciprocal ratio consensus and it is the first time that ratio strategy is combined with second-order DTMASs. Furthermore, our work covers several cases where dynamically changing topologies are directed or undirected, connected or disconnected, or even mixed patterns of connected and disconnected graphs.

Distributed Optimization for Second-Order Discrete-Time Multiagent Systems With Set Constraints.

The optimization problem of second-order discrete-time multiagent systems with set constraints is studied in this article. In particular, the involved agents cooperatively search an optimal solution of a global objective function summed by multiple local ones within the intersection of multiple constrained sets. We also consider that each pair of local objective function and constrained set is exclusively accessible to the respective agent, and each agent just interacts with its local neighbors. By borrowing from the consensus idea, a projection-based distributed optimization algorithm resorting to an auxiliary dynamics is first proposed without interacting the gradient information of local objective functions. Next, by considering the local objective functions being strongly convex, selection criteria of step size and algorithm parameter are built such that the unique solution to the concerned optimization problem is obtained. Moreover, by fixing a unit step size, it is also shown that the optimization result can be relaxed to the case with just convex local objective functions given a properly chosen algorithm parameter. Finally, practical and numerical examples are taken to verify the proposed optimization results.

Event-based bipartite consensus of second-order discrete-time multi-agent systems with matrix-weighted signed networks

This article researches the bipartite consensus for discrete-time second-order multi-agent systems on matrix-weighted signed networks, which can describe the inter-dependencies of multidimensional states among states. So as to save limited communication resources, based on the matrix-weighted combined measurements of the position and velocity states, a matrix-weighted event-triggered control algorithm is designed. With the help of the stability theory, variable transformation and the inequality technique, the bipartite consensus conditions which are based on coupling gains, discrete interval, the parameters in the event-triggered rule and communication topology are obtained. Furthermore, the conditions to avoid the controller updating in each discrete-time are supplied. At last, a simulation example is offered to demonstrate the theoretical results.

Leader-Following Consensus of Discrete-Time Stochastic Nonlinear Multiagent Systems Under Fixed and Switching Topologies via Impulsive Control

In this article, the leader-following consensus of discrete-time second-order stochastic nonlinear multiagent systems with Markovian switching topology is investigated. Assume that the switching rule between the communication topologies is subject to the average dwell time (ADT) constraint. By taking advantage of the impulsive control strategy, the leader-following consensus issue is transformed into the stability problem of the impulsive error systems. Several sufficient conditions respecting the exponential mean-square stability for impulsive error system under fixed and switching topologies are derived by utilizing the Lyapunov function and the notions of average impulsive interval and ADT. Intriguingly, this article considers that impulsive instant and switching instant are independent of each other. Four potential cases of system evolution under Markovian switching topology are discussed. Finally, simulation results are reported to illustrate the effectiveness of our developed results.

Distributed Model-Free Sliding-Mode Predictive Control of Discrete-Time Second-Order Nonlinear Multiagent Systems With Delays.

In this article, the tracking problem of networked discrete-time second-order nonlinear multiagent systems (MASs) is studied. First, for the MASs without communication delay, a novel method, called distributed model-free sliding-mode control algorithm is proposed, which can make the system converge quickly without the accurate model. Furthermore, for the MASs with delay, in order to eliminate the influence of time delay on the system, a distributed model-free sliding-mode predictive control strategy based on time-delay compensation technology is proposed, which can actively compensate for time delay while ensuring system stability and consensus tracking performance requirements. Both the simulation and experiment results reveal the superiority of the proposed methods.

Necessary and Sufficient Conditions for Leader–Follower Consensus of Discrete-Time Multiagent Systems With Smart Leader

In this article, the leader-follower consensus is investigated for discrete-time second-order multiagent systems with smart leader, where the smart leader can obtain the feedback information from its neighbours. Different from the existing results, an objective function is provided to determine whether the leader utilizes the feedback information from its neighboring agents, with the purpose of making the system have better performance. Under the directed interconnected topology, a novel distributed consensus protocol is designed for second-order discrete-time multiagent systems. Some necessary and sufficient conditions are derived for the leader-follower consensus of discrete-time second-order multiagent systems with smart leader under fixed/switching topology. Finally, two simulation examples are presented to demonstrate the feasibility of the theoretical results.

Optimal consensus control for unknown second-order multi-agent systems: Using model-free reinforcement learning method

In this paper, the optimal consensus control problem with second-order dynamics consisting of leader and follower agents is discussed. For optimal consensus problem, the optimal control policies rely on algebraic Riccati equations (AREs) equation, which are difficult to solve. Furthermore, both the follower agents’ and the leader agent’s dynamics are assumed to be completely unknown. As the consensus problem based on feedback control, the second-order discrete-time multi-agent systems (DT-MASs) model with directed topology is formulated to the optimal tracking control problem via online deep reinforcement learning method. Based on graph theory, matrix analysis, Lyapunov stability, deep learning and optimal control, the optimality of value function and the stability of the consensus error systems for the unknown second-order systems are guaranteed for each agent. The results show that the designed policy iteration algorithm not only stabilizes the distributed dynamic systems, but also makes all agents’ position and velocity states reach consensus, respectively. Finally, the correctness of our theoretical results is illustrated under two numerical simulations based on the designing model-free actor-critic networks.

An event-triggered consensus protocol for quantized second-order multi-agent systems with network delay and process noise

This paper focuses on the quantized consensus problem of discrete-time second-order multi-agent systems which suffer from bounded process noise. In such systems, neighboring agents exchange information through a digital network that has only a finite bit rate and is subject to bounded network delay. Network delay may exacerbate the effects of quantization error and degrade the control performance by reducing the information to be extracted from receiving time instants of exchange packets. To handle network delay and save communication bit rates, an event-triggered distributed control protocol based on a dynamic encoding–decoding policy is proposed. Under the proposed event-triggered protocol, the asymptotic consensus of the concerned noise-free multi-agent systems can be achieved at a lower bit rate than that of periodic sampling control. This bit rate superiority results from the fact that our protocol makes use of the extra information being extracted from the event-triggering time instants. Moreover, it is shown that no higher bit rate is required to guarantee the desired consensus in the existence of bounded process noise. Simulations are done to demonstrate the effectiveness of our consensus protocol. As shown by our numerical examples, the occupied bit rate to ensure the desired consensus can be saved by up to 50% compared with periodic sampling strategies.

Constrained H∞ Consensus of Second-Order Discrete-Time Multi-Agent Networks With Nonconvex Velocity Constraints

This paper addresses the constrained H <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">∞</sub> consensus of second-order discrete-time multi-agent systems with nonconvex velocity constraints and external disturbances over a directed network. A novel nonlinear distributed control law is proposed to enable all agents converge to an agreement cooperatively while their velocities remain in different nonconvex constraint sets. To measure the effect of external disturbances on consensus, a nonlinear H <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">∞</sub> output function is introduced via maximum and minimum functions. By a model transformation, the original closed-loop system is changed into an equivalent one. And then, the convergence analysis and H <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">∞</sub> performance analysis are completed by utilizing Lyapunov stability theory and robust control theory. In addition, the sufficient constrained H <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">∞</sub> consensus conditions are deduced in the form of matrix inequalities under the assumption of strong connectivity of directed network. Finally, numerical simulations are provided to illustrate the effectiveness of the theoretical results.

Bipartite containment control for discrete-time second-order multiagent systems with time-varying delays on switching signed topologies

Compared with the traditional cooperative networks, the signed networks containing both cooperative and competitive interactions have prevailed. In light of such consideration, the problem of bipartite containment control for second-order multiagent systems (MASs) including multiple leaders over signed networks is investigated in this paper, in which time-varying delays and switching topologies are considered. Firstly, an augmented system method is explored such that the bipartite containment problem is formulated as the convergence problem of the product of an infinite number of sub-stochastic matrices. Then, by constructing a new leader–follower topology for each switching topology, the corresponding relationships between two topologies are used to simplify our model. Moreover, based on the properties of non-negative matrix and the technique of edge composition, a mild topological condition is developed for guaranteeing the bipartite containment collective behavior. Finally, numerical examples are presented for verifying the correctness of our conclusion.

Time‐varying formation of second‐order discrete‐time multi‐agent systems under non‐uniform communication delays and switching topology with application to UAV formation flying

In this study, the time-varying formation control problem for the second-order discrete-time multi-agent systems is investigated, where both the non-uniform communication time-delays and switching topology are taken into account. A linear discrete-time formation protocol is developed based on the neighbouring relative position and velocity information. Using the state transformation method and properties of the stochastic matrix, the formation feasibility condition is given and sufficient conditions for the discrete-time multi-agent systems to accomplish the time-varying formation are established. An unmanned aerial vehicles (UAVs) formation experiment platform is constructed. Using four quadrotor UAVs, UAV formation flying experiments are performed to verify the effectiveness and reliability of the discrete-time formation protocol. The experimental results show that the theoretical results can be used to deal with the time-varying formation control problem for multiple UAVs system with communication delays and switching topology.

Position-constrained containment for second-order discrete-time multi-agent systems

In this paper, we study a containment control problem for second-order multi-agent systems in the presence of convex position constraints and switching graphs. A distributed algorithm with a nonlinear projection operator is proposed to guarantee that each agent eventually converges into the convex hull spanned by some given multiple static points, while the position of each agent is kept lying in a given closed set. It is proved that all agents ultimately converge into the convex hull while each agent remains in its constraint set as long as there exists a directed path from the static points to the agents in the union of the graphs in each bounded time interval. Finally, a numerical example is given to illustrate the obtained theoretical results.

Scaled Tracking Consensus in Discrete-Time Second-Order Multiagent Systems With Random Packet Dropouts

This article focuses on the issue of scaled tracking consensus for discrete-time second-order multiagent systems under random packet dropouts, where the cases with a static leader and a dynamic leader are considered, respectively. The scaled tracking consensus means that all agents reach a consensus value determined by the leader but with different scales, and the phenomenon of packet dropout on each communication link is described as a Bernoulli variable independent of other communication links. By virtue of random environment-based scaled consensus algorithms, it is shown how to reconstruct the original system into augmented error systems with random coefficient matrices. With the kind assistance of substochastic matrix and super-stochastic matrix, sufficient conditions for the cases with a static leader and a dynamic leader are derived, respectively. Moreover, computer simulations are performed to demonstrate the dynamics of network agents under random packet dropouts.

Containment Problem for Multiagent Systems With Nonconvex Velocity Constraints.

In this paper, the containment problem with nonconvex velocity constraints is studied for second-order discrete-time multiagent systems. A distributed projection-based algorithm is proposed for all followers to be gathered in the convex area formed by multiple stationary leaders. It is revealed that the algorithm can solve the velocity-constrained containment problem with communication delays and switching networks provided that each follower has at least a directed path from some leaders to itself in the union of graphs. The main analysis approaches include model transformation techniques, Lyapunov function, and convexity analysis. A simulation example is also included to elucidate the obtained result.

An analysis on containment control for discrete-time second-order multi-agent systems with asynchronous intermittent communication

In this paper, a containment control issue for discrete-time second-order multi-agents (MASs) with asynchronous intermittent communication is analyzed, where each agent communicates with some of its neighbors at certain discrete time according to its own clock rather than the complete discrete process and all agents’ clocks are independent of each other. By constructing the spanning subgraph of communication topology at each discrete time, the stability analysis of containment control under the asynchronous intermittent communication is equivalently transformed into the products convergence of nonnegative matrices. Hybrid tools including matrix theory and the composite binary relation are utilized to analyze this convergence issue. Unlike the existing results on the containment control with fixed topology, in which the followers will gradually enter into the convex hull constructed by the leaders and their final positions are fixed, it is theoretically proved in this paper that although the followers will enter into the convex hull of the leaders under the asynchronous intermittent communication, their positions after entering the convex hull may always be time-varying. Finally, the theoretical finding is verified by a simulation example.

Time-Varying Formation Control for Second-Order Discrete-Time Multi-Agent Systems With Directed Topology and Communication Delay

Time-varying formation control protocol design and analysis problems for the second-order discrete-time multi-agent systems with directed interaction topology and communication delay are investigated. A local information-based distributed protocol is designed by utilizing the delayed state information of neighbors. Through system decomposition and stability analysis, an explicit description of the feasible time-varying formation set is given. Necessary and sufficient conditions for the systems with the directed topology and communication delay to achieve time-varying formation are obtained, which are related to the topology of the interaction graph and the feasibility of the predefined formation. Necessary constraints on the gain parameters and the sampling period are proposed, so as to guide the design of parameters in the protocol. The numerical simulation results indicate that the protocol can steer the agents to accomplish the desired time-varying formation and effectively tolerate the relatively large bounded communication delay. Outdoor experiment with quadrotors is presented to demonstrate the effectiveness of the obtained theoretical results with one sampling period delay.

Time-Varying Formation Control For Second-Order Discrete-Time Multi- Agent Systems With Switching Topologies and Nonuniform Communication Delays

Time-varying formation control and velocity tracking problems for second-order discrete-time multi-agent systems with switching jointly-connected topologies and nonuniform communication delays are investigated. A local information-based distributed protocol is designed by utilizing the instantaneous states of the agent itself and the delayed states of its neighbors. Through model transformation and stability analysis, an explicit mathematical description of a feasible time-varying formation set is proposed. Necessary and sufficient conditions for the systems with switching topologies and nonuniform communication delays to achieve the feasible time-varying formation are obtained. The coupling constraints on the gain parameters and sampling period are proposed, so as to guide the design of parameters in the protocol. We include the effects of velocity tracking error damping gain in the protocol and derive milder conditions which allow for not only bounded nonuniform communication delays but also for dynamically switching directed graphs that are jointly connected. The numerical examples are further presented to illustrate the validity and effectiveness of the obtained results.