Optimized leader–follower consensus control for high-order nonlinear multi-agent system modeled in canonical dynamic form

Abstract

This paper addresses the optimized leader–follower consensus control problem of nonlinear canonical dynamic multi-agent system (MAS). Since every agent is modeled in high-order dynamic, they contain the various state variables having derivative relations. Consequently, to attain the optimized consensus control, backstepping technique and reinforcement learning (RL) are combined together. In the first backstepping step, the virtual control is formulated to have the consensus error term consisting of neighbor agents’ output states. In the last backstepping step, since the nonlinearity dynamic is arisen, the optimized actual control is derived by performing the critic-actor RL. It is worth mentioning that, in the traditional RL optimal control, the critic and actor updating laws are obtained by performing the gradient descent algorithm to square of the approximated Hamilton–Jacobi-Bellman (HJB) equation, which includes multiple nonlinearity items, as a result, the algorithm is very intricate. However, in this optimized consensus control, since the RL training laws are yielded in the light of negative gradient of a simple positive function that is relevant to the HJB equation, its algorithm is very simple. At the same time, it can also avoid the persistence excitation conditions. Finally, theory and simulation confirm the validity of this optimized consensus method.