Discrete-time Multi-agent Systems Research Articles

Iterative learning consensus control for switched discrete-time multi-agent systems based on the 2-D linear discrete model

In this article, a class of switched nonlinear discrete-time heterogeneous (NDTH) multi-agent systems (MASs) with switching signals in both agent dynamics and communication topologies are considered, and a distributed iterative learning control (ILC) law is proposed for the output consensus of the switched NDTH MAS so that the outputs of all follower agents can well track the leader agent. With the proposed ILC controller, the iterative learning dynamic process of the switched NDTH MAS is firstly formulated as a two-dimensional (2-D) linear discrete Roesser model with specified boundary states at each switching sub-interval. Afterwards, according to the exploited properties on solution of the 2-D linear discrete Roesser model, a sufficient convergence theorem for the presented ILC controller is derived. At last, the validity of the ILC-based consensus controller is illustrated through a simulation experiment.

Distributed optimization consensus for multi-agent systems on matrix-weighted networks.

In this paper, the distributed optimization consensus issues for both first-order continuous time (CT) and discrete-time (DT) multi-agent systems (MASs) on matrix-weighted networks are studied. In order to make each agent achieve optimization consensus, a new matrix-weighted distributed optimization algorithm for CT and DT MASs is proposed. Using the Lyapunov stability theory and matrix theory, the optimization consensus conditions are obtained, respectively. Finally, the correctness of our results is verifiied by numerical examples.

Distributed optimal consensus control for discrete-time multi-agent systems with composite switching topologies via an iterative Q-learning method

This paper aims to achieve distributed optimal consensus control for discrete-time multi-agent systems (MASs) under composite switching topologies through utilising the iterative Q-learning approach. First, the optimal consensus control strategy, independent of system dynamics information, is developed by using the designed iterative Q-learning approach to guarantee that all follower states can synchronise to the desired trajectory formulated by the leader. Compared with the invariable algebraic or edge-fixed topology structures, the designed Q-learning control algorithm in this paper is implemented based on the composite switching topologies consisting of periodic and stochastic mechanisms. Then, an error dynamic system is established, which transforms the optimal consensus control issue into an optimal regulation issue and removes the influence of the two types of switching topologies. To conquer the problems of lack of model information and the inaccessible analytical solution of Hamilton-Jacobi-Isaacs (HJI) equation, an iterative Q-learning approach is designed according to the reconstructed Q-function Bellman equation. Through further iterative learning, the optimal control strategy can be acquired so as to realise consensus control and ensure the stability of the closed-loop systems. Finally, two simulation examples are provided to demonstrate the effectiveness of the proposed distributed optimal control scheme.

Data-driven distributed output consensus control for multi-agent systems with unknown internal state

This paper discusses the topic of optimal output consensus control of linear time-invariant discrete-time multi-agent systems (MASs). The attainment of optimal output consensus control for MASs hinges upon the resolution of the interconnected Hamilton–Jacobi-Bellman equation, a task typically precluded by the inherent intractability of analytical solutions. Furthermore, most real-world systems are too complex to obtain the internal states of systems. To address these issues, a modified deep Q-learning network is constructed using current and historical system data rather than a precise model of the system. First, reconstructing the internal state of each agent using an adaptive distributed observer based on output feedback prevents the system instability brought on by the augmented systems. Then the local error system of the agent can be redefined. Based on the redefined error system, a data-driven adaptive dynamic programming (ADP) method is introduced, realized by using the actor–critic neural network structure. In addition, an experience replay strategy is proposed to reduce the propagation of estimation bias and improve the learning speed. Finally, the comparative numerical simulations substantiate the efficacy of the proposed algorithm in a quantifiable manner.

Radial basis function neural network based data-driven iterative learning consensus tracking for unknown multi-agent systems

This paper provides a novel data-driven-distributed-consensus control protocol for unknown nonlinear non-affine discrete-time multi-agent systems (MAS) with repetitive properties. The leader’s commands are directed to the followers in the topological graph. The dynamic linearization technology (DLT) is used to build the distributed iterative learning (IL) controller along the iteration axis. In the iterative process, the control gain is automatically adjusted by updating the weight matrix of the high-order radial basis function neural network (RBFNN, HORBFNN). In global control, the higher order parameter (HOP) Newton method is used to achieve global convergence and stability of the control process. All the above processes do not require the understanding of dynamical equations or physical models for each agent, and only use local communication information of multi-agent to achieve consistent tracking of MAS leaders and followers. Based on the strong connection, the convergence performance, stability and boundedness properties of the proposed control protocol in the fixed topology as well as in the iterative topology are validated by a rigorous theoretical analysis. Simulation experiments are conducted to verify the effectiveness of the control protocol.

Resilient Consensus for Discrete-Time Multiagent Systems With a Dynamic Leader and Time Delay: Theory and Experiment.

The ever-present cyber-attacks have posed a significant challenge to consensus of multiagent systems (MASs), due to their capability of compromising agents. These threats underscore the critical need to design control strategies that can endow MASs with resilience. In this article, we address resilient consensus problems for first- and second-order discrete-time MASs, considering the presence of malicious agents, a dynamic leader, and communication delay. First, a resilient controller is designed for first-order MASs, and sufficient conditions are derived based on existing robust graph concepts to achieve consensus with ultimately bounded error. Next, since the derived error bound grows factorially with the system scale, a novel graph structure and a modified controller are proposed to limit the growth to a linear rate. Building upon the obtained results, an estimator-based control framework is introduced to solve resilient consensus problems for second-order MASs. Finally, comparative simulations and practical experiments based on unmanned ground vehicles and unmanned-aerial-vehicles are conducted to validate the effectiveness and practicability of the proposed control methods.

Min–max group consensus of discrete-time multi-agent systems under directed random networks

This paper studies the min–max group consensus of discrete-time multi-agent systems under a directed random graph, where the presence of each directed edge is randomly determined by a probability and independent of the presence of other edges. Firstly, we propose a min–max consensus protocol without memory, and give the necessary and sufficient conditions to ensure that the multi-agent system can achieve the min–max group consensus in the sense of almost sure and mean square, respectively. Secondly, we design a novel consensus protocol with memory and a behavior mechanism. Using the stochastic analysis theory and the extremal algebra, some necessary and sufficient conditions are obtained for achieving the min–max group consensus in the sense of almost sure and mean square, respectively. It is shown that the protocol with memory can solve the loss problem of the maximum and minimum initial states. Finally, the effectiveness of the two group consensus protocols and the behavior mechanism is verified by four numerical simulations.

Consensus of Discrete-Time Multiagent Systems With Transient Performance Constraints

Consensus of Discrete-Time Multiagent Systems With Transient Performance Constraints

Two-Layer Reinforcement Learning for Output Consensus of Multiagent Systems Under Switching Topology.

In this article, the data-based output consensus of discrete-time multiagent systems under switching topology (ST) is studied via reinforcement learning. Due to the existence of ST, the kernel matrix of value function is switching-varying, which cannot be applied to existing algorithms. To overcome the inapplicability of varying kernel matrix, a two-layer reinforcement learning algorithm is proposed in this article. To further implement the proposed algorithm, a data-based distributed control policy is presented, which is applicable to both fixed topology and ST. Besides, the proposed method does not need assumptions on the eigenvalues of leader's dynamic matrix, it avoids the assumptions in the previous method. Subsequently, the convergence of algorithm is analyzed. Finally, three simulation examples are provided to verify the proposed algorithm.

Encryption–decryption-based distributed fault-tolerant consensus tracking control for multi-agent systems

This study investigates fault-tolerant consensus tracking for discrete-time multi-agent systems (MASs) subject to external eavesdropping threats and additive actuator faults. First, actuator faults are modeled by difference equations, and decentralized observers are constructed to estimate actuator faults as well as system states. To offset fault-induced effects, ensure secure communication, and alleviate communication congestion, neighboring encrypted state information based on the encryption–decryption strategy (EDS) and estimated fault are integrated into a distributed active fault-tolerant consensus tracking control (FCTC) protocol. Through the properties of compatible norms, criteria for the controller, observer, and dynamic encryption key in EDS are derived to achieve leader-following consensus (LFC) of MASs with bias and drift actuator faults. Simulation results confirm the validity of the encryption–decryption-based distributed FCTC strategy.

Asynchronous iterative Q-learning based tracking control for nonlinear discrete-time multi-agent systems

This paper addresses the tracking control problem of nonlinear discrete-time multi-agent systems (MASs). First, a local neighborhood error system (LNES) is constructed. Then, a novel tracking algorithm based on asynchronous iterative Q-learning (AIQL) is developed, which can transform the tracking problem into the optimal regulation of LNES. The AIQL-based algorithm has two Q values QiA and QiB for each agent i, where QiA is used for improving the control policy and QiB is used for evaluating the value of the control policy. Moreover, the convergence of LNES is given. It is shown that the LNES converges to 0 and the tracking problem is solved. A neural network-based actor-critic framework is used to implement AIQL. The critic network of AIQL is composed of two neural networks, which are used for approximating QiA and QiB respectively. Finally, simulation results are given to verify the performance of the developed algorithm. It is shown that the AIQL-based tracking algorithm has a lower cost value and faster convergence speed than the IQL-based tracking algorithm.

Differentially Private Consensus Control for Discrete-Time Multiagent Systems: Encoding–Decoding Schemes

Differentially Private Consensus Control for Discrete-Time Multiagent Systems: Encoding–Decoding Schemes

Fault Detection and Fault-Tolerant Control for Discrete-Time Multiagent Systems With Sensor Faults: A Data-Driven Method

Fault Detection and Fault-Tolerant Control for Discrete-Time Multiagent Systems With Sensor Faults: A Data-Driven Method

Optimizing control efficiency in discrete-time multi-agent systems via event-triggered containment techniques combining disturbance handling and input delay management

The goal of this paper is to mitigate disturbances and input delays while optimizing controller actuation updates for discrete-time multi-agent systems through the use of an event-triggered confinement control system, especially in resource-constrained scenarios. This approach when combined with event-triggered control techniques, then every follower in the system adjusts its condition at specified times based on an event-triggered condition that is suggested. The containment control system issue in the presence of disturbances and input delays was tackled by using both decentralized and centralized event-triggered control systems. Using matrix theory and the Lyapunov technique, convergence analysis is conducted to show that the proposed strategy stays free of zeno phenomena. Numerical boosts are used to further illustrate the impact of theoretical results.

Decentralized leader-following consensus control design for discrete-time multi-agent systems with switching topology

The problem of consensus control of linear discrete-time multi-agent systems (MASs) with switching topology is considered in the presence of a leader. The goal of consensus control is to bring the states of all agents to the leader state while providing stability for local agents, as well as the MAS as a whole. In contrast to the traditional approach, which uses the concept of an extended dynamic multi-agent system model and communication topology graph Laplacian, this paper proposes a decomposition approach, which provides a separate design of local controllers. The control law is chosen in the form of distributed feedback with discrete PID controllers. The problem of local controllers’ design is reduced to a set of semidefinite programming problems using the method of invariant ellipsoids. Sufficient conditions for agents’ stabilization and global consensus condition fulfillment are obtained using the linear matrix inequality technique. The availability of information about a finite set of possible configurations between agents allows us to design local controllers offline at the design stage. A numerical example demonstrates the effectiveness of the proposed approach.

Data-Driven Optimal Bipartite Consensus Control for Second-Order Multiagent Systems via Policy Gradient Reinforcement Learning.

This article investigates the optimal bipartite consensus control (OBCC) problem for unknown second-order discrete-time multiagent systems (MASs). First, the coopetition network is constructed to describe the cooperative and competitive relationships between agents, and the OBCC problem is proposed by the tracking error and related performance index function. Based on the distributed policy gradient reinforcement learning (RL) theory, a data-driven distributed optimal control strategy is obtained to guarantee the bipartite consensus of all agents' position and velocity states. In addition, the offline data sets ensure the learning efficiency of the system. These data sets are generated by running the system in real time. Besides, the designed algorithm is an asynchronous version, which is essential to solve the challenge caused by the computational ability difference between nodes in MASs. Then, by means of the functional analysis and Lyapunov theory, the stability of the proposed MASs and the convergence of the learning process are analyzed. Furthermore, an actor-critic structure containing two neural networks is used to implement the proposed methods. Finally, a numerical simulation shows the effectiveness and validity of the results.

Data-driven output consensus for a class of discrete-time multiagent systems by reinforcement learning techniques

The primary objective of this research paper is to examine the challenges associated with optimization and tracking control in discrete-time multiagent systems. This study begins by defining the optimal tracking control issue within the framework of a leader–follower multiagent system. A novel approach based on policy iteration is introduced for calculating the control law and performance index. The paper also includes an analysis of the convergence properties of this technique, ensuring its effectiveness in reaching optimal solutions. To facilitate this, an actor–critic neural network framework is utilized to approximate the control law and iterative performance index function. The proposed approach enables the real-time implementation of the policy iteration method, eliminating the requirement for prior knowledge of the system’s dynamics. The paper concludes with a series of simulation experiments that demonstrate the practicality and efficiency of the developed optimal tracking control strategies.

Data-based adaptive predictive bipartite consensus for nonlinear multiagent systems against DoS attacks

This article investigates a Denial-of-Service (DoS) attack problem for nonlinear unknown discrete-time multiagent systems (MASs) to implement bipartite consensus tracking tasks with fixed and switching topologies. Firstly, an equivalent linearization data model of each agent is constructed using a pseudo partial derivative approach, where only one parameter needs to be estimated using input/output data of the controlled MASs. Meanwhile, the DoS attack behavior is described by a Bernoulli distribution process, and both cooperative and competitive relationships among agents are investigated. Moreover, an increment prediction compensator is designed to reduce the effect of DoS attacks. A data-based adaptive predictive bipartite consensus control algorithm is formulated. The corresponding theoretical analysis indicates that tracking errors of MASs with fixed and switching topologies converge to a small range around zero. Finally, several simulations and hardware tests further verify the proposed scheme’s effectiveness.

Max-consensus of multi-agent systems in random networks

This paper considers max-consensus of a discrete-time multi-agent system (MAS) in directed random networks. Interactions among agents in the MAS are probabilistic and independent with each other. By using max-plus algebra and random theory, a sufficient and necessary condition is given for achieving max-consensus of the MAS. Moreover, we demonstrate that the max-consensus in four probabilistic senses (almost surely, in probability, expectation and mean square) is equivalent when expected graph is strongly connected. This ensures that max-consensus can be achieved in multi-agent systems even if random failures occur in the communication network, which is of practical importance in the fields of wireless sensor networks and distributed computing. A simulation example is presented to illustrate the effectiveness of theoretical results.

Matrix-Weighted Consensus of Second-Order Discrete-Time Multiagent Systems.

In this article, we study the matrix-weighted consensus issues for second-order discrete-time multiagent systems on directed network topology. Under the designed matrix-weighted consensus algorithm, based on the eigenvalues of the Laplacian matrix, coupling gains, and discrete interval, we build some consensus conditions for reaching discrete-time consensus and deduce some simplified and straightforward consensus conditions for undirected network topology. Besides, for a given network topology, we theoretically analyze the influence of the coupling gains and discrete intervals on the consensus conditions of the network dynamics. Finally, we offer several simulation examples to validate the obtained results.