Unknown Nonlinear Multi-agent Systems Research Articles

Optimal Tracking Control of Nonlinear Multiagent Systems Using Internal Reinforce Q-Learning.

In this article, a novel reinforcement learning (RL) method is developed to solve the optimal tracking control problem of unknown nonlinear multiagent systems (MASs). Different from the representative RL-based optimal control algorithms, an internal reinforce Q-learning (IrQ-L) method is proposed, in which an internal reinforce reward (IRR) function is introduced for each agent to improve its capability of receiving more long-term information from the local environment. In the IrQL designs, a Q-function is defined on the basis of IRR function and an iterative IrQL algorithm is developed to learn optimally distributed control scheme, followed by the rigorous convergence and stability analysis. Furthermore, a distributed online learning framework, namely, reinforce-critic-actor neural networks, is established in the implementation of the proposed approach, which is aimed at estimating the IRR function, the Q-function, and the optimal control scheme, respectively. The implemented procedure is designed in a data-driven way without needing knowledge of the system dynamics. Finally, simulations and comparison results with the classical method are given to demonstrate the effectiveness of the proposed tracking control method.

Distributed optimized dynamic event-triggered control for unknown heterogeneous nonlinear MASs with input-constrained

The distributed optimized dynamic event-triggered controller is investigated for completely unknown heterogeneous nonlinear multi-agent systems (MASs) on a directed graph subject to input-constrained. First, the distributed observer is designed to estimate the information of the leader for each follower, and a network of the augmented system is constructed by employing the dynamics of the followers and the observers. An identifier with a compensator is designed to approximate the unknown augmented system (agent) with an arbitrarily small identifier error. Then, consider that the input-constrained optimal controller, along with Hamilton–Jacobi–Bellman (HJB) equation, is under pressure to execute in certain systems associated with bottlenecks such as communication and computing burdens. A critic–actor-based optimized dynamic event-triggered controller, which tunes the parameters of critic–actor neural networks (NNs) by the dynamic triggering mechanism, is leveraged to determine the rule of aperiodic sampling and maintain the desired synchronization service. In addition, the existence of a positive minimum inter-event time (MIET) between consecutive events is also proved. Finally, the applications in non-identical nonlinear MAS and 2-DOF robots illustrate the availability of the proposed theoretical results.

Coordinated Fuzzy Adaptive Iterative Learning Control of Consensus for Unknown Nonlinear Multi-agent Systems

Coordinated Fuzzy Adaptive Iterative Learning Control of Consensus for Unknown Nonlinear Multi-agent Systems

Distributed Fuzzy Adaptive Output-Feedback Control of Unknown Nonlinear Multiagent Systems in Strict-Feedback Form.

This article is concerned with the cooperative tracking control problem for heterogeneous multiagent systems in a leader-following form under a directed graph. The dynamics of each following agent is unknown, obeying a strict-feedback form. With the help of fuzzy-logic systems, input filters, and constraint-handling schemes, a fully distributed output-feedback control algorithm is proposed to achieve output synchronization with prescribed performance and guarantee boundedness of signals in the closed-loop systems. In addition, the algorithm exhibits a simplicity control attribute in the sense that: 1) the control design utilizes only relative output measurements, and no extra information needs to be transmitted via the network and 2) the issue of explosion of complexity is addressed, without employing command filters or dynamic surface control techniques. Finally, the simulation results clarify and verify the established theoretical findings.

Data‐driven consensus control for a class of unknown nonlinear multiagent systems with time delays

In this paper, the distributed output consensus tracking problem for a class of unknown heterogeneous discrete-time single-input single-output nonlinear multiagent systems (MASs) with time delays is considered. For this problem, a distributed data-driven consensus control strategy is proposed by using improved model-free adaptive control with a time delay. First, the unknown nonlinear multiagent time delay system is transformed into a virtual data model with time-varying parameters by using a partial form dynamic linearisation method. Then, a controller with model-free adaptive control and a tracking differentiator are designed. The proposed method uses only the input/output data of neighbouring agents during the control process without mathematically modelling multiple agents. For the time delay system, the boundedness of the closed-loop tracking error is theoretically proven for both the tracking problem and the regulation problem. Furthermore, this result can be generalised to switching topology structures by a rigorous mathematical proof. Numerical simulations verify the effectiveness of the proposed algorithm.

Neural network-based adaptive synchronization for second-order nonlinear multiagent systems with unknown disturbance.

This paper handles the distributed adaptive synchronization problem for a class of unknown second-order nonlinear multiagent systems subject to external disturbance. It is supposed to be an unknown one for the underlying external disorder. First, the neural network-based disturbance observer is developed to deal with the impact induced by the strange disturbance. Then, a new distributed adaptive synchronization criterion is put forward based on the approximation capability of the neural networks. Next, we propose the necessary and sufficient condition on the directed graph to ensure the synchronization error of all followers can be reduced small enough. Then, the distributed adaptive synchronization criterion is further explored because it is difficult to obtain the relative velocity measurements of the agents. The distributed adaptive synchronization criterion without the velocity measurement feedback is also designed to fulfill the current investigation. Finally, the simulation example is performed to verify the correctness and effectiveness of the proposed theoretical results.

Optimal Synchronization Control of Heterogeneous Asymmetric Input-Constrained Unknown Nonlinear MASs via Reinforcement Learning

The asymmetric input-constrained optimal synchronization problem of heterogeneous unknown nonlinear multiagent systems (MASs) is considered in the paper. Intuitively, a state-space transformation is performed such that satisfaction of symmetric input constraints for the transformed system guarantees satisfaction of asymmetric input constraints for the original system. Then, considering that the leader's information is not available to every follower, a novel distributed observer is designed to estimate the leader's state using only exchange of information among neighboring followers. After that, a network of augmented systems is constructed by combining observers and followers dynamics. A nonquadratic cost function is then leveraged for each augmented system (agent) for which its optimization satisfies input constraints and its corresponding constrained Hamilton-Jacobi-Bellman (HJB) equation is solved in a data-based fashion. More specifically, a data-based off-policy reinforcement learning (RL) algorithm is presented to learn the solution to the constrained HJB equation without requiring the complete knowledge of the agents' dynamics. Convergence of the improved RL algorithm to the solution to the constrained HJB equation is also demonstrated. Finally, the correctness and validity of the theoretical results are demonstrated by a simulation example.

Point-to-point consensus tracking control for unknown nonlinear multi-agent systems using data-driven iterative learning

This paper considers the point-to-point consensus tracking control for a class of nonlinear multi-agent systems with completely unknown dynamics, where the consensus is concerned with some given desired points instead of the entire desired trajectory. It is assumed that the multi-agent system executes repetitive coordination tasks in a finite time interval and the iterative learning control is also utilized to design a consensus protocol with learning ability. To deal with the unknown nonlinear agent’s dynamic, the relationship between agent’s output at these given points and agent’s control input is first derived and then a data-based model referring to the agent’s dynamic is established by utilizing the iteration-domain dynamical linearization technique. Then, a data-driven iterative learning protocol is developed by optimizing two performance indexes, which contains a control input updated algorithm, a parameter estimation algorithm and a reset algorithm. The results show that the proposed design can achieve the point-to-point consensus tracking task only by using the I/O data of the agent. Finally, simulation examples are provided to verify the effectiveness of the proposed protocol.

Adaptive consensus control for a class of nonlinear multi-agent systems with unknown time delays and external disturbances

This paper investigates a class of unknown nonlinear multi-agent systems with unknown time delays and external disturbances. The external disturbance considered in this paper is completely unknown, and the accurate mathematical expression cannot be obtained. To eliminate the external disturbance, a disturbance observer is constructed by the Disturbance-Observer-Based-Control (DOBC) method. By utilizing Young’s inequality scaling technique, the problem of time delay is solved. By assuming that the communication graph among the agents is directed and connected, a novel distributed adaptive consensus control protocol is established based on the relative output information to achieve the goal of consensus control. Furthermore, the stability of the multi-agent system is analyzed by selecting an appropriate Lyapunov function; it is certified that with the proposed control protocol, all signals are guaranteed to be globally uniformly bounded, and the considered multi-agent systems can reach consensus even in the presence of time delay and external disturbance. Finally, two simulation examples are given to support the proposed control strategy.

A novel bipartite consensus tracking scheme for unknown nonlinear multi-agent systems: Theoretical analysis and applications

This article proposes a novel distributed data-driven bipartite consensus tracking scheme for bipartite consensus tracking problems of multi-agent systems with bounded disturbances and coopetition networks. The proposed scheme only uses the input/output data of each agent without requiring the agents’ dynamics. We obtain the equivalent dynamic linearization data model for a controlled plant using the dynamic linearization technique based on the pseudo partial derivative. Considering the cooperative and competitive interactions among agents, the proposed method ensures that agents with adversarial relationships implement bipartite consensus tracking tasks even if only a subset of agents can access the information from the virtual leader. Moreover, the strict proof process of convergence properties reveals that the tracking error coverages to a small range around the origin. We also establish a set of software and hardware platform to demonstrate the effectiveness of the proposed distributed data-driven bipartite consensus tracking method.

Quantized model-free adaptive iterative learning bipartite consensus tracking for unknown nonlinear multi-agent systems

This paper considers the data quantization problem for a class of unknown nonaffine nonlinear discrete-time multi-agent systems (MASs) under repetitive operations to achieve bipartite consensus tracking. Here, a quantized distributed model-free adaptive iterative learning bipartite consensus control (QDMFAILBC) approach is proposed based on the dynamic linearization technology, algebraic graph theory, and sector-bound methods. The proposed approach doesn’t require each agent’s dynamics knowledge and only uses the input/output data of MASs, where the data is coded by the logarithmic quantizer before being transmitted. Moreover, we consider both cooperative and competitive relationships among agents. We rigorously prove the stability of the proposed scheme and analyze the effects of data quantization. Meanwhile, we demonstrate that data quantization does not affect the stability of MASs, and bipartite consensus tracking errors can converge to zero with the processing of the proposed scheme, although the data quantization slows the convergence rate. Furthermore, the results are extended to switching topologies, and three simulation studies further validate the effectiveness of the designed method.

Data-driven tracking consensus for a class of unknown nonlinear multi-agent systems

This study investigates the tracking consensus problem for a class of unknown nonlinear multi-agent systems A novel data-driven protocol for this problem is proposed by using the model-free adaptive control method To obtain faster convergence speed, one-step-ahead desired signal is introduced to construct the novel protocol Here, switching communication topology is considered, which is not required to be strongly connected all the time Through rigorous analysis, sufficient conditions are given to guarantee that the tracking errors of all agents are convergent under the novel protocol Examples are given to validate the effectiveness of results derived in this article

Adaptive consensus for heterogeneous unknown nonlinear multi-agent systems with asymmetric input dead-zone: A finite-time approach

This study deals with the adaptive finite-time consensus problem of heterogeneous multi-agent systems composed of first-order and second-order agents with unknown nonlinear dynamics and asymmetric input dead-zone under connected undirected topology. Under the proposed protocol and adaptive laws, a sliding mode variable for every agent converges to a compact set in finite time, and also the position errors and the velocity errors (for second-order agents) between any two agents converge to a small desired neighborhood of the origin in finite time. Each agent requires its states and the relative positions of its neighbors. By applying sliding mode control, the external disturbances, and the imperfect approximation of neural networks are rejected. The unknown terms of the agents’ dynamics are approximated using radial basis function neural networks. The adaptive compensator plus dead-zone is applied to overcome the asymmetric input dead-zone. Based on Lyapunov stability theory, analysis is led on stability. Different from the previous works, the global information graph is not used in the proposed protocol. Finally, our approach is examined for two examples to evaluate its performance.

Asymptotic Synchronization Control of High-order Nonlinear Multiagent Systems Using Barrier Functions

This paper studies the adaptive control problem for unknown time-varying high-order nonlinear multiagent systems (MASs) with directed topology. A novel low-complexity distributed adaptive backstepping controller is developed to achieve the asymptotic synchronization. The designed controller regards the unknown system non-linearities as “disturbance-like” terms, which are guaranteed to be bounded by using the barrier functions, such that detail models of system nonlinearities are released. Then, the “disturbance-like” terms are compensated adaptively by designing the novel compensator at each step, such that the synchronization error is eliminated to zero eventually for each agent. In addition, unlike the existing backstepping-like synchronization approaches for high-order nonlinear MASs, the “explosion of complexity” issue is avoided without extra low-pass filters. Some simulations are shown to demonstrate the effectiveness and advantages of the developed method.

Neural Network Observer Based Consensus Control of Unknown Nonlinear Multi-agent Systems with Prescribed Performance and Input Quantization

This paper investigates the consensus tracking problem with predefined performances requirements for a class of unknown nonlinear multi-agent systems with hysteresis quantizer and external disturbances under a directed graph topology. Neural network observers are designed to estimate unmeasurable states and the the consensus tracking problem with performance requirements is transformed to a stabilization problem by prescribed performance error transformation schemes. The novel consensus protocol can be applied to a more general class of nonlinear multi-agent systems since the Lipschitz condition is avoided and state information is not required. It is strictly proved that all signals in the closed-loop systems are cooperatively uniformly ultimately bounded and both the transient and steady performances of the consensus tracking satisfy prescribed performance requirements. Finally, two numerical examples are presented to validate the effectiveness of the proposed strategy.

Predefined-Time Adaptive Neural-Network-Based Consensus Tracking Control for Nonlinear Multiagent Systems With Zero Tracking Error

This paper is dedicated to addressing the predefined-time consensus tracking control problem for unknown high-order nonlinear multiagent systems. The prominent difference compared with some existing papers is that the follower is modeled in the form of non-strict-feedback structure. Meanwhile, instead of being constants, the real control gains are unknown functions. A distinct advantage in our work is that the outputs of followers are able to track the output of leader within the time specified in advance. In order to get our desired predefined-time controller, radial basis function (RBF) neural networks (NNS) are applied to compensate those unknown nonlinearities. Then in the design framework of adaptive backstepping, the predefined-time virtual control laws are presented and their derivatives are approximated by using finite-time differentiators. Under our proposed predefined-time controller, it is rigorously demonstrated that the whole closed-loop system remains stable and all outputs of the followers track the reference signal in predefined time. In the end, a simulation example is given to ulteriorly verify the efficacy of the suggested predefined-time control scheme.

Distributed Adaptive Bipartite Time-Varying Formation Control for Heterogeneous Unknown Nonlinear Multi-Agent Systems

This paper investigates the bipartite time-varying formation control problem for a class of heterogeneous unknown nonlinear multi-agent systems. This bipartite time-varying formation is composed of two sub-formations whose relationship is opposite. By parameterization the unknown nonlinear items of all agents, a distributed adaptive proportional-integral protocol is presented under undirected signed topology to ensure the convergence of the bipartite time-varying formation error. Some sufficient conditions for choosing suitable control parameters are given. In the case of systems with external time-varying but bounded disturbances, the uniformly ultimate boundedness of the formation error and its integration can be guaranteed by the proposed control protocol. A simulative example is provided to illustrate the effectiveness of the proposed protocol.

Distributed synchronisation control of unknown nonlinear systems with an active leader

This paper investigates the optimal synchronisation problem for unknown nonlinear heterogeneous multi-agent systems (MASs) with an active leader and disturbances. Due to the external input signal of the active leader, the resulted uncooperative behaviours and unpredictability increase difficulty to derive the analytical solution of the coupled Bellman equation. To this end, we develop a distributed model reference adaptive critic learning algorithm with a hierarchical and distributed synchronisation control framework, which transforms the synchronisation problem of unknown nonlinear MASs into that of defined reference ones. Moreover, the algorithm extends our previous work to the distributed MASs, so that it is guaranteed that all the agents reach an agreement with the minimised transient errors and robustness against the disturbances and the neural network (NN) approximation errors. It is proved that the local neighbour tracking errors and the NN weight estimation errors are uniformly ultimately bounded via Lyapunov stability theory. A simulation example for multi-manipulator systems demonstrates the effectiveness of the algorithm.

Finite-time consensus of heterogeneous unknown nonlinear multi-agent systems with external disturbances via event-triggered control

This article investigates the practical finite-time consensus for a class of heterogeneous multi-agent systems composed of first-order and second-order agents with heterogeneous unknown nonlinear dynamics and external disturbances in an undirected communication topology. To reduce the system updates, we propose an event-triggered approach. By defining auxiliary states, an adaptive distributed event-triggered control is designed to achieve practical finite-time consensus. Unknown nonlinear dynamics for each agent are estimated using radial basis function neural network. The stability of the overall closed-loop system is studied through the Lyapunov criterion. It is proven that by applying the proposed control scheme, the local neighbor position error and the velocity error between any two agents converge to a small region in finite time. Furthermore, it is shown that the Zeno behavior is ruled out. Finally, applicability and effectiveness of the proposed control scheme is verified and validated by two examples.

Globally repetitive learning consensus control of unknown nonlinear multi-agent systems with uncertain time-varying parameters

This paper studies the globally repetitive learning leader-following consensus for a class of unknown nonlinear multi-agent systems with uncertain periodic time-varying parameters. Neural networks are used as the feed-forward compensators to model some unknown nonlinear system functions. According to the different distributions of agents in the network topology, and by learning periodic uncertainties, a new hybrid repetitive learning control protocol is presented based on Lyapunov theory. Meanwhile, the proposed learning control protocol can guarantee that all the followers can track the leader asymptotically. Moreover, the boundedness of all the signals is ensured. Finally, three simulation examples are provided to verify the effectiveness of the proposed scheme.