Optimal Consensus Control Research Articles

A Novel Parallel Control Method for Optimal Consensus of Nonlinear Multiagent Systems.

This work concentrates on the initial introduction of parallel control to investigate an optimal consensus control strategy for continuous-time nonlinear multiagent systems (MASs) via adaptive dynamic programming (ADP). First, the control input is integrated into the feedback system for parallel control, facilitating an augmented system's optimal consensus control with an appropriate augmented performance index function to be established, which is identical to the original system's suboptimal control with a conventional performance index. Second, the feasibility of the proposed control scheme is evaluated based on the policy iteration algorithm, and the convergence of the algorithm is demonstrated. Then, an online learning algorithm becomes available to implement the ADP-based optimal parallel consensus control protocol without prior knowledge of the system. The Lyapunov approach is employed to indicate that the signals are convergent. Ultimately, the experimental data support the theoretical results.

Optimal consensus control of dynamically interconnected multi-agent systems: A SDRE approach for efficient and stable operation

This study focuses on achieving optimal consensus control based on the state-dependent Riccati equation (SDRE) in multi-agent systems (MASs) characterized by the interconnectivity of multiple sub-systems with physical interconnections. The objective is to establish a suitable communication topology among the MAS controllers that considers both local and global objectives while ensuring stability through distributed control. To address this, an innovative extension of leader-follower and leaderless consensus control methodologies is proposed, accompanied by stability analysis employing the Lyapunov method. Additionally, the paper explores consensus control for dynamically interconnected MASs handling suspended load transportation, such as multi-lift systems. By utilizing the principles of virtual work by D'Alembert and Gauss's principle of least constraint, a constrained multibody system model is developed, incorporating the Udwadia-Kalaba motion equation approach. Simulation results and theoretical analysis validate the efficiency and performance of the proposed consensus controller structure.

Optimal Consensus Control for Continuous-Time Linear Multiagent Systems: A Dynamic Event-Triggered Approach.

This article investigates the optimal consensus problem for general linear multiagent systems (MASs) via a dynamic event-triggered approach. First, a modified interaction-related cost function is proposed. Second, a dynamic event-triggered approach is developed by constructing a new distributed dynamic triggering function and a new distributed event-triggered consensus protocol. Consequently, the modified interaction-related cost function can be minimized by applying the distributed control laws, which overcomes the difficulty in the optimal consensus problem that seeking the interaction-related cost function needs all agents' information. Then, some sufficient conditions are obtained to guarantee optimality. It is shown that the developed optimal consensus gain matrices are only related to the designed triggering parameters and the desirable modified interaction-related cost function, relaxing the constraint that the controller design requires the knowledge of system dynamics, initial states, and network scale. Meanwhile, the tradeoff between optimal consensus performance and event-triggered behavior is also considered. Finally, a simulation example is provided to verify the validity of the designed distributed event-triggered optimal controller.

Observer-based reinforcement learning for optimal fault-tolerant consensus control of nonlinear multi-agent systems via a dynamic event-triggered mechanism

In this article, an adaptive optimized consensus tracking control problem is studied for nonlinear strict-feedback dynamic multi-agent systems (MASs), considering both unmeasurable system states and time-varying bias faults. By utilizing the backstepping technique, we develop an adaptive reinforcement learning (RL) algorithm within the observer-critic-actor architecture, specially designed to compensate for the lack of state information and derive control inputs, thereby achieving approximate optimal control. Moreover, an event-triggered mechanism is introduced in the sensor-to-controller channel, which dynamically adjusts the triggering threshold online and employs event-sampled states to initiate control actions. To address discontinuities caused by state triggering, we construct virtual controllers that continuously sample state signals and reconfigure the actual controller based on previously triggered states. The outputs of the MASs are shown to accurately track the desired reference signals while ensuring the boundedness of all closed-loop signals. Additionally, the proposed controller is verified to be devoid of Zeno behavior. Finally, the effectiveness of our control methodology is demonstrated through numerical simulation.

Dynamic event-driven optimal consensus control for state-constrained multiagent zero-sum differential graphical games

In this paper, a dynamic event-driven optimal control scheme is proposed for the zero-sum differential graphical games in nonlinear multiagent systems with full-state constraints. Initially, to address the dual demands of optimality and state constraints, a set of system transformation functions are introduced to satisfy the state constraints of the agents. Then, by applying the principle of differential game theory, the distributed optimal control problem affected by external disturbances is formulated as a zero-sum differential graphical game, and the performance index function related to neighbor informations and disturbances is designed for each follower. Afterwards, to enhance the utilization of communication resource, a novel dynamic event-triggered mechanism characterized by a dynamic threshold parameter and an auxiliary dynamic variable is developed, which not only exhibits greater flexibility but also diminishes the frequency of triggers. Furthermore, the approximate optimal control strategies are obtained by employing an event-driven adaptive dynamic programming algorithm. Ultimately, a simulation example is presented to verify the applicability of the proposed control approach.

Fault-Tolerant Optimal Consensus Control for Heterogeneous Multi-Agent Systems

This study explores fault-tolerant consensus in leader–following heterogeneous multi-agent systems, focusing on actuator failures in uncrewed aerial vehicles (UAVs) and uncrewed ground vehicles (UGVs). An optimization-based fault-tolerant consensus algorithm is proposed. The algorithm utilizes the Euler–Lagrange formula to ensure system consistency under actuator failures, with the Lyapunov stability theory proving the asymptotic stability of the consistency error. The algorithm is applied to heterogeneous multi-agent systems of UAVs and UGVs to derive optimal fault-tolerant consensus control laws for each vehicle type. Simulation experiments give evidence for the feasibility of the proposed control strategy.

Action-dependent heuristic dynamic programming for hybrid-order systems based on dynamic event-triggered mechanism

We investigate the distributed optimal consensus control problem based on action-dependent heuristic dynamic programming (ADHDP). In order to strike a stable balance between the learning cost of reinforcement learning and the resource utilization efficiency of the hybrid-order multi-agent systems (MASs), we propose an improved dynamic event-triggered ADHDP (dET-ADHDP) method. This approach can non-periodically explore the control policy distribution using the online action-dependent actor–critic (ADAC) learning framework. Meanwhile, it can dynamically adjust the trigger lower bound by exploiting the designed trigger threshold function, and adaptively decide the signal trigger moment during the ADAC learning process. In addition, we demonstrate the boundedness of the ADAC network weights and show that under the designed dynamic event-triggering rules, the MASs can asymptotically achieve optimal tracking control without Zeno phenomenon. Finally, compared with the traditional static counterparts, simulation experiments demonstrate that the proposed dynamic event-triggered ADAC (dET-ADAC) algorithm has more efficient resource utilization while maintaining satisfactory learning performance.

Research on Multi-UAV Obstacle Avoidance with Optimal Consensus Control and Improved APF

To address collision challenges between multi-UAVs (unmanned aerial vehicles) during obstacle avoidance, a novel formation control method is proposed. Leveraging the concept of APF (artificial potential field), the proposed approach integrates UAV maneuver constraints with a consensus formation control algorithm, optimizing UAV velocities through the particle swarm optimization (PSO) algorithm. The optimal consensus control algorithm is then employed to achieve the optimal convergence rate of the UAV formation. To mitigate the limitations of traditional APF, a collinear force deflection angle is introduced, along with an obstacle avoidance method aimed at preventing UAVs from being trapped in locally optimal solutions. Additionally, an obstacle avoidance algorithm based on virtual force fields between UAVs is designed. Comparative analysis against the basic algorithm demonstrates the effectiveness of the designed optimal consensus algorithm in improving formation convergence performance. Moreover, the improved APF resolves local optimal solution issues, enabling UAVs to effectively navigate around obstacles. Simulation results validate the efficacy of this method in achieving multi-UAV formation control while effectively avoiding obstacles.

Data-Driven Optimal Bipartite Consensus Control for Second-Order Multiagent Systems via Policy Gradient Reinforcement Learning.

This article investigates the optimal bipartite consensus control (OBCC) problem for unknown second-order discrete-time multiagent systems (MASs). First, the coopetition network is constructed to describe the cooperative and competitive relationships between agents, and the OBCC problem is proposed by the tracking error and related performance index function. Based on the distributed policy gradient reinforcement learning (RL) theory, a data-driven distributed optimal control strategy is obtained to guarantee the bipartite consensus of all agents' position and velocity states. In addition, the offline data sets ensure the learning efficiency of the system. These data sets are generated by running the system in real time. Besides, the designed algorithm is an asynchronous version, which is essential to solve the challenge caused by the computational ability difference between nodes in MASs. Then, by means of the functional analysis and Lyapunov theory, the stability of the proposed MASs and the convergence of the learning process are analyzed. Furthermore, an actor-critic structure containing two neural networks is used to implement the proposed methods. Finally, a numerical simulation shows the effectiveness and validity of the results.

Optimal bipartite consensus control for heterogeneous unknown multi-agent systems via reinforcement learning

This study focuses on addressing optimal bipartite consensus control (OBCC) problems in heterogeneous multi-agent systems (MASs) without relying on the agents' dynamics. Motivated by the need for model-free and optimal consensus control in complex MASs, a novel distributed scheme utilizing reinforcement learning (RL) is proposed to overcome these challenges. The MAS network is randomly partitioned into sub-networks where agents collaborate within each subgroup to attain tracking control and ensure convergence of positions and speeds to a common value. However, agents from distinct subgroups compete to achieve diverse tracking objectives. Furthermore, the heterogeneous MASs considered have unknown first and second-order dynamics, adding to the complexity of the problem. To address the OBCC issue, the policy iteration (PI) algorithm is used to acquire solutions for discrete-time Hamilton-Jacobi-Bellman (HJB) equations while implementing a data-driven actor-critic neural network (ACNN) framework. Ultimately, the accuracy of our proposed approach is confirmed through the presentation of numerical simulations.

Data-Based Optimal Consensus Control for Multiagent Systems With Time Delays: Using Prioritized Experience Replay

Data-Based Optimal Consensus Control for Multiagent Systems With Time Delays: Using Prioritized Experience Replay

Adaptive Optimal Consensus Control of Multiagent Systems With Unknown Dynamics and Disturbances via Reinforcement Learning

Adaptive Optimal Consensus Control of Multiagent Systems With Unknown Dynamics and Disturbances via Reinforcement Learning

Optimal Consensus Control for Multi-Agent Systems With Unknown Dynamics and States of Leader: A Distributed KREM Learning Method

Optimal Consensus Control for Multi-Agent Systems With Unknown Dynamics and States of Leader: A Distributed KREM Learning Method

Model‐free distributed optimal control for general discrete‐time linear systems using reinforcement learning

AbstractThis article proposes a novel data‐driven framework of distributed optimal consensus for discrete‐time linear multi‐agent systems under general digraphs. A fully distributed control protocol is proposed by using linear quadratic regulator approach, which is proved to be a sufficient and necessary condition for optimal control of multi‐agent systems through dynamic programming and minimum principle. Moreover, the control protocol can be constructed by using local information with the aid of the solution of the algebraic Riccati equation (ARE). Based on the Q‐learning method, a reinforcement learning framework is presented to find the solution of the ARE in a data‐driven way, in which we only need to collect information from an arbitrary follower to learn the feedback gain matrix. Thus, the multi‐agent system can achieve distributed optimal consensus when system dynamics and global information are completely unavailable. For output feedback cases, accurate state information estimation is established such that optimal consensus control is realized. Moreover, the data‐driven optimal consensus method designed in this article is applicable to general digraph that contains a directed spanning tree. Finally, numerical simulations verify the validity of the proposed optimal control protocols and data‐driven framework.

Reinforcement learning-based consensus control for MASs with intermittent constraints

In this article, an adaptive optimal consensus control problem is studied for multiagent systems in the strict-feedback structure with intermittent constraints (the constraints appear intermittently). More specifically, by designing a novel switch-like function and an improved coordinate transformation, the constrained states are converted into unconstrained states, and the problem of intermittent constraints is resolved without requiring “feasibility conditions”. In addition, using the composite learning algorithm and neural networks to construct the identifier, a simplified identifier-actor-critic-based reinforcement learning strategy is proposed to obtain the approximate optimal controller under the framework of backstepping. Meanwhile, with the aid of the nonlinear dynamic surface control technique, the issue of “explosion of complexity” in backstepping is removed, and the requirements for filter parameters are loosened. Based on Lyapunov stability theory, it is demonstrated that all signals in the closed-loop system are bounded. Finally, two simulation examples are used to verify the effectiveness of the proposed method.

Event-Based Robust Optimal Consensus Control for Nonlinear Multiagent System With Local Adaptive Dynamic Programming.

This article investigates the robust optimal consensus for nonlinear multiagent systems (MASs) through the local adaptive dynamic programming (ADP) approach and the event-triggered control method. Due to the nonlinearities in dynamics, the first part defines a novel measurement error to construct a distributed integral sliding-mode controller, and the consensus errors can approximately converge to the origin in a fixed time. Then, a modified cost function with augmented control is proposed to deal with the unmatched disturbances for the event-based optimal consensus controller. Specifically, a single network local ADP structure with novel concurrent learning is presented to approximate the optimal consensus policies, which guarantees the robustness of the MASs and the uniform ultimate boundedness (UUB) of the neural network (NN) weights' estimation error and relaxes the requirement of initial admissible control. Finally, an illustrative simulation verifies the effectiveness of the method.

Distributed Fault Tolerant Consensus Control of Nonlinear Multiagent Systems via Adaptive Dynamic Programming.

This article develops a distributed fault-tolerant consensus control (DFTCC) approach for multiagent systems by using adaptive dynamic programming. By establishing a local fault observer, the potential actuator faults of each agent are estimated. Subsequently, the DFTCC problem is transformed into an optimal consensus control problem by designing a novel local value function for each agent which contains the estimated fault, the consensus errors, and the control laws of the local agent and its neighbors. In order to solve the coupled Hamilton-Jacobi-Bellman equation of each agent, a critic-only structure is established to obtain the approximate local optimal consensus control law of each agent. Moreover, by using Lyapunov's direct method, it is proven that the approximate local optimal consensus control law guarantees the uniform ultimate boundedness of the consensus error of all agents, which means that all following agents with potential actuator faults synchronize to the leader. Finally, two simulation examples are provided to validate the effectiveness of the present DFTCC scheme.

Adaptive Optimal Bipartite Consensus Control for Heterogeneous Multi-Agent Systems

Adaptive Optimal Bipartite Consensus Control for Heterogeneous Multi-Agent Systems

Optimal consensus control of discrete-time stochastic multi-agent systems

Optimal consensus control of discrete-time stochastic multi-agent systems

Distributed Optimal Consensus Control of Constrained Multi-Agent Systems: A Non-Separable Optimization Perspective

Distributed Optimal Consensus Control of Constrained Multi-Agent Systems: A Non-Separable Optimization Perspective