Optimal Consensus Problem Research Articles

Distributed optimal consensus control for nonlinear multi-agent systems with input saturation based on event-triggered adaptive dynamic programming method

In this paper, the distributed optimal consensus problem is investigated for a class of continuous-time nonlinear multi-agent systems with input saturation. Non-quadratic cost functions are introduced to handle input constraints and a novel distributed optimal consensus protocol is derived based on event-triggered adaptive dynamic programming method. An online implement scheme is designed under actor-critic network framework in order to obtain the solutions of Hamilton-Jacobi-Bellman equations online. The computation and communication loads are effectively reduced since the weight estimation vectors and controllers are updated only at event-triggered instants. Detailed analysis is presented based on Lyapunov stability theory which guarantees that the weight estimation errors and local consensus errors are uniformly ultimately bounded. Furthermore, it proves that Zeno behaviour can be effectively avoided. Finally, the simulation examples are presented to validate the proposed strategy.

Randomized optimal consensus of multiagent systems based on a novel intermittent projected subgradient algorithm

SummaryIn this article, a novel intermittent projected subgradient algorithm is presented to solve the randomized optimal consensus problem for heterogeneous multiagent systems with time‐varying communication topologies. The multiagent systems achieve the consensus meanwhile minimizing the global objective function via the proposed algorithm, where fi(x) is the convex objective function of agent i itself. Due to the common Bernoulli distribution adopted in the existing random optimization algorithm without considering the different computing capability of each agent. An individual projection probability is assigned for each agent based on computing capabilities so that either making projection or taking average is chosen according to the above probability which can effectively avoid overload for some agents with lower computing capabilities and improve the reliability of the overall systems. A new sufficient step‐size condition is given to ensure all agents converge to the optimal solution with probability one. Finally, a numerical example is also given to validate the proposed method.

Optimal Output Consensus for Nonlinear Multiagent Systems With Both Static and Dynamic Uncertainties

In this technical note, we investigate an optimal output consensus problem for heterogeneous uncertain nonlinear multi-agent systems. The considered agents are described by high-order nonlinear dynamics subject to both static and dynamic uncertainties. A two-step design, comprising sequential constructions of optimal signal generator and distributed partial stabilization feedback controller, is developed to overcome the difficulties brought by nonlinearities, uncertainties, and optimal requirements. Our study can not only assure an output consensus, but also achieve an optimal agreement characterized by a distributed optimization problem.

Optimal Consensus-Based Distributed Control Strategy for Coordinated Operation of Networked Microgrids

A two-layer optimal consensus-based distributed control strategy is proposed for the coordinated operation of networked microgrids (MGs). Several networked MG operation objectives are optimized using the proposed control strategy, including frequency/voltage regulation, proportional active power sharing, and smooth MG islanding/reconnection operation. The global objective function of networked MGs is decomposed into a series of local objectives assigned to participating distributed energy resources (DERs). In the decomposed optimization, each control layer is realized in a distributed manner in which DER state variables reach consensus along with optimizing their assigned local objective functions. Each control layer in the two-layer control strategy is formulated as an optimal consensus problem in which the optimality and asymptotical stability of the system equilibrium point for each control layer are demonstrated. The effectiveness of the proposed control strategy is validated in a modified IEEE 33-bus distribution system using the PSCAD/EMTDC platform.

Optimal Consensus Control for Heterogeneous Nonlinear Multiagent Systems with Partially Unknown Dynamics

This paper focuses on an optimal consensus problem for heterogeneous discrete-time nonlinear multi-agent systems (MASs) with partially unknown dynamics. For those systems, it is difficult to obtain the solution of the coupled Hamilton-Jacobi-Bellman (HJB) equations, which is necessary to address the optimal consensus problem. A new hierarchical and distributed optimal control strategy is developed to derive the near solution of the HJB equations. Its control structure consists of the model reference adaptive control (MRAC) layer and distributed control layer. In the MRAC layer, the adaptive feedforward and feedback controller is designed to make the states of followers converge to ones of their corresponding reference models. Then, the optimal consensus problem of heterogeneous MASs is formulated as that of homogeneous MASs. In the distributed control layer, an online distributed value iteration algorithm is proposed to approximate the optimal solution of the HJB equations for reference models. Thereby, the optimal consensus is also achieved for the heterogeneous MASs. The two convergence properties are analyzed to demonstrate the MRAC performance and the optimal consensus, respectively. Simulation results verify the effectiveness of the proposed strategy.

Distributed Optimal Consensus Control Algorithm for Continuous-Time Multi-Agent Systems

This brief proposes a distributed algorithm to solve the optimal consensus control problem of continuous-time multi-agent systems. With the definition of consensus manifold, the utility function of the optimization problem is formulated as a linear quadratic polynomial which indicates the tradeoff between convergence rate and energy cost in a finite horizon. Applying the alternating direction method of multiplier, the optimal consensus problem is separated into three subproblems: 1) an input optimization problem; 2) a consensus state optimization problem; and 3) a dual optimization problem, thereby separating the communication topology and the agents dynamics. Then, these subproblems can be analyzed and solved independently, following from which that a distributed control algorithm is proposed to allow the continuous-time multi-agent systems to reach optimal consensus. A numerical example is given to demonstrate the effectiveness of the proposed algorithm.

Distributed optimal consensus control for nonlinear multiagent systems based on DISOPE algorithm

SummaryIn this note, a distributed dynamic integrated system optimization and parameter estimation (DISOPE) algorithm for nonlinear multiagent system with general local finite time horizon performance index is proposed, which integrates the model optimization and parameter estimation techniques. Two difficult issues for optimal consensus problem, nonlinear multiagent system and nonlinear quadratic performance index, are considered. With the help of DISOPE algorithm, a distributed optimal control policy is designed to minimize the local performance index in finite time horizon. Furthermore, the convergence of distributed DISOPE algorithm is given, such that the convergence solution satisfies the optimality conditions. Finally, a simulation is employed to show the effectiveness of the distributed DISOPE algorithm.

Distributed optimal consensus algorithms in multi-agent systems

This paper investigates the distributed optimal consensus problem by designing the multi-agent communication systems, where each agent has local access to its own strongly convex utility function. Here, we design two distributed consensus-based optimization algorithms to tackle the optimization problem of multi-agent systems. One is the second-order optimization consensus algorithm, and another is the third-order algorithm. Moreover, the two algorithms can ensure that all agents achieve consensus at a common state which minimizes the sum of the objective function under the assumption that interaction graph is strongly connected and detailed balanced. For each of the proposed algorithms, it can be found that the equilibrium point of the algorithm is the optimal point and it is proved to be asymptotically stable under given conditions. Finally, the performance of the proposed algorithms are verified by two simulations based on an IEEE 14-bus test system and an IEEE 57-bus test system.

Distributed optimal consensus of multiple double integrators under bounded velocity and acceleration

This paper studies a distributed optimal consensus problem for multiple double integrators under bounded velocity and acceleration. Assigned with an individual and private convex cost which is dependent on the position, each agent needs to achieve consensus at the optimum of the aggregate cost under bounded velocity and acceleration. Based on relative positions and velocities to neighbor agents, we design a distributed control law by including the integration feedback of position and velocity errors. By employing quadratic Lyapunov functions, we solve the optimal consensus problem of double-integrators when the fixed topology is strongly connected and weight-balanced. Furthermore, if an initial estimate of the optimum can be known, then control gains can be properly selected to achieve an exponentially fast convergence under bounded velocity and acceleration. The result still holds when the relative velocity is not available, and we also discuss an extension for heterogeneous Euler-Lagrange systems by inverse dynamics control. A numeric example is provided to illustrate the result.

Constrained optimal consensus in dynamical networks

In this paper, an optimal consensus problem with local inequality constraints is studied for a network of single-integrator agents. The goal is that a group of single-integrator agents rendezvous at the optimal point of the sum of local convex objective functions. The local objective functions are only available to the associated agents that only need to know their own positions and of their neighbors. This point is supposed to be confined by some local inequality constraints. To tackle this problem, we integrate the primal-dual gradient-based optimization algorithm with a consensus protocol to drive the agents toward the agreed point that satisfies KKT conditions. The asymptotic convergence of the solution of the optimization problem is proven with the help of LaSalle's invariance principle for hybrid systems. A numerical example is presented to show the effectiveness of our protocol.

Model-Free Optimal Consensus Control of Networked Euler-Lagrange Systems

This paper considers the model-free optimal consensus problem of networked Euler-Lagrange systems without velocity measurements. By employing the position information, a novel neural network-based velocity observer is built for each agent to estimate the unmeasurable velocity vector and unknown system model. Based on the estimated velocity information, we propose distributed optimal control policies depended on the solutions to the coupled Hamilton-Jacobi-Bellman (HJB) equations. Then, a model-free policy iteration (PI) algorithm is provided to learn the coupled HJB equations online. To implement the PI algorithm, the critic-action neural networks are built and their weights are updated based on the gradient descent method. The uniform ultimate boundedness of the integrated observer estimation errors, the integrated consensus errors, and the weight estimation errors for the observer-critic-action neural networks is demonstrated by the Lyapunov technique. Finally, the numerical simulation on a directed network with six nonlinear manipulators is presented to validate the theoretical results.

Data-Based Optimal Control of Multiagent Systems: A Reinforcement Learning Design Approach.

This paper studies an optimal consensus tracking problem of heterogeneous linear multiagent systems. By introducing tracking error dynamics, the optimal tracking problem is reformulated as finding a Nash-equilibrium solution to multiplayer games, which can be done by solving associated coupled Hamilton-Jacobi equations. A data-based error estimator is designed to obtain the data-based control for the multiagent systems. Using the quadratic functional to approximate every agent's value function, we can obtain the optimal cooperative control by the input-output (I/O) Q -learning algorithm with a value iteration technique in the least-square sense. The control law solves the optimal consensus problem for multiagent systems with measured I/O information, and does not rely on the model of multiagent systems. A numerical example is provided to illustrate the effectiveness of the proposed algorithm.

Global optimal consensus for discrete-time multi-agent systems with bounded controls

In this paper, we consider the global optimal consensus problem for discrete-time multi-agent systems with bounded control protocols over a fixed and directed communication network. Each agent is described by a discrete-time single integrator and endowed with a quadratic objective function which is private to itself. For each agent, we develop two bounded distributed protocols: bounded proportional–integral (PI) protocol and bounded integral (I) protocol, based on the information received from its neighboring agents through the communication network and the gradient of its own objective function. We show that the proposed bounded distributed protocols with properly chosen parameters solve the global optimal consensus problem, i.e., the multi-agent system achieves consensus at a state that minimizes the sum of the objective functions, if the directed communication network is strongly connected and detailed balanced.

Constrained optimal consensus in multi-agent systems with single- and double-integrator dynamics

ABSTRACTThis paper fully studies distributed optimal consensus problem in undirected dynamical networks. We consider a group of networked agents that are supposed to rendezvous at the optimal point of a collective convex objective function. Each agent has no knowledge about the global objective function and only has access to its own local objective function, which is a portion of the global one, and states information of agents within its neighbourhood set. In this setup, all agents coordinate with their neighbours to seek the consensus point that minimises the network's global objective function. In the current paper, we consider agents with single-integrator and double-integrator dynamics. Further, it is supposed that agents' movements are limited by some convex inequality constraints. In order to find the optimal consensus point under the described scenario, we combine the interior-point optimisation algorithm with a consensus protocol and propose a distributed control law. The associated convergence analysis based on Lyapunov stability analysis is provided.

Optimal Output Consensus of High-Order Multiagent Systems With Embedded Technique.

In this paper, we study an optimal output consensus problem for a multiagent network with agents in the form of multi-input multioutput minimum-phase dynamics. Optimal output consensus can be taken as an extended version of the existing output consensus problem for higher-order agents with an optimization requirement, where the output variables of agents are driven to achieve a consensus on the optimal solution of a global cost function. To solve this problem, we first construct an optimal signal generator, and then propose an embedded control scheme by embedding the generator in the feedback loop. We give two kinds of algorithms based on different available information along with both state feedback and output feedback, and prove that these algorithms with the embedded technique can guarantee the solvability of the problem for high-order multiagent systems under standard assumptions.

Receding horizon consensus of general linear multi-agent systems with input constraints: An inverse optimality approach

This paper investigates the optimal consensus problem for general linear MASs (of semi-stable and unstable dynamics) subject to control input constraints. The optimal consensus protocols are first designed by inverse optimality approach, based on which the centralized receding horizon control (RHC)-based consensus strategies are designed and the feasibility and consensus properties of the closed-loop systems are analyzed. Utilizing the centralized one, distributed RHC-based consensus strategies are developed. We show that (1) the optimal performance indices under the inverse optimal consensus protocols are coupled with the network topologies and the system matrices of subsystems; (2) the unstable modes of subsystems impose more stringent requirements for the parameter design; (3) the designed RHC-based consensus strategies can make the control input constraints fulfilled and ensure convergent consensus and consensus for MASs with semi-stable and unstable subsystems, respectively.

Necessary and sufficient conditions for distributed constrained optimal consensus under bounded input

SummaryIn this paper, we study a distributed constrained optimal consensus problem for discrete‐time first‐order integrator systems under bounded input. Each agent is assigned with a local convex cost function, and all agents are required to achieve consensus at the minimum of the aggregate cost over a common convex constraint set, which is only accessible by part of the agents. A 2‐step control protocol is designed to solve the problem under bounded input. Firstly, at each time step, each agent moves a bounded step of the subgradient descent from an individual cost and another one along the projection direction if it can access the constraint. The second movement is then given by a bounded average of the relative positions to neighbors. Specifically, to coordinate the subgradient step within the network without using global information, we introduce an estimate of the upper bound of all agents' subgradients, which is updated by a unilateral consensus mechanism. Under the given control protocol, we obtain the necessary and sufficient conditions to achieve the constrained optimal consensus for a fixed topology. Under similar conditions, we also solve the problem for switching topologies and conduct a convergence rate analysis for strongly convex costs.

Distributed Optimal Consensus for Multi-Agent Systems Under Independent Position and Velocity Topology

In this paper, linear quadratic regulator (LQR) theory is applied to solve the inverse optimal consensus problem for a second-order linear multi-agent systems (MAS) under independent position and velocity topology. The optimal Laplacian matrices related to the topologies of position and velocity are derived by solving the algebraic Riccati equation (ARE). Theoretically, we obtain the optimal Laplacian matrices, which correspond to the directed strongly connected graphs, for the second-order multi-agent systems. Finally, two simulation examples are provided to verify the theoretical analysis of this paper.

Minimum time state consensus for cooperative attack of multi-missile systems

This paper considers the cooperative attack with multiple missiles on a stationary target and a two-step procedure is proposed. First the states of all missiles are guided to achieve minimum time consensus, and then, by using the proportional navigation guidance law with identical navigation gain each missile arrives at the target simultaneously or sequentially. Among which, the strategy to realize state consensus in minimum time is the focus of the issue. In the paper the coordination algorithms are developed based on bang–bang control strategy due to its simply structure, subsequently, it is proved that the time optimal consensus problem can be transformed into a parameter optimization problem. Besides, the proposed schemes are able to significantly reduce the number of parameters to be optimized and finally solve the problem with little computation cost. Different from the impact time control methods, the strategy for state consensus does not demand the information of time-to-go. Meanwhile only the initial values of missiles' states are required for the coordination algorithms which lead to less communication burden. Two different examples are provided to demonstrate the validity and efficiency of our proposed strategy.

Data-Driven Optimal Consensus Control for Discrete-Time Multi-Agent Systems With Unknown Dynamics Using Reinforcement Learning Method

This paper investigates the optimal consensus control problem for discrete-time multi-agent systems with completely unknown dynamics by utilizing a data-driven reinforcement learning method. It is known that the optimal consensus control for multi-agent systems relies on the solution of the coupled Hamilton–Jacobi–Bellman equation, which is generally impossible to be solved analytically. Even worse, most real-world systems are too complicated to obtain accurate mathematical models. To overcome these deficiencies, a data-based adaptive dynamic programming method is presented using the current and past system data rather than the accurate system models also instead of the traditional identification scheme which would cause the approximation residual errors. First, we establish a discounted performance index and formulate the optimal consensus problem via Bellman optimality principle. Then, we introduce the policy iteration algorithm which motivates this paper. To implement the proposed online action-dependent heuristic dynamic programming method, two neural networks (NNs), 1) critic NN and 2) actor NN, are employed to approximate the iterative performance index functions and control policies, respectively, in real time. Finally, two simulation examples are provided to demonstrate the effectiveness of the proposed method.