Optimized Formation Control Using Simplified Reinforcement Learning for a Class of Multiagent Systems With Unknown Dynamics

Abstract

The article proposes an optimized leader-follower formation control using a simplified reinforcement learning (RL) of identifier-critic-actor architecture for a class of nonlinear multiagent systems. In general, optimal control is expected to be obtained by solving Hamilton-Jacobi-Bellman (HJB) equation, but the equation associated with a nonlinear system is difficult to solve by analytical method. Although the difficulty can be effectively overcome by the RL strategy, the existing RL algorithms are very complex because their updating laws are obtained by carrying out gradient descent algorithm to square of the approximated HJB equation (Bellman residual error). For a multiagent system, due to the state coupling problem, these methods will become difficult implementation. In the proposed optimized scheme, the RL updating laws are derived from negative gradient of a simple positive function, which is the equivalence to HJB equation; therefore, the control algorithm is significantly simple. Furthermore, in order to solve the problem of unknown system dynamics, an adaptive identifier is integrated into the control. Finally, the theory and simulation demonstrate that the optimized formation scheme can guarantee the desired control performance.