Decentralized Adaptive Optimal Tracking Control for Massive Multi-Agent Systems with Input Constraint

Abstract

In this paper, the decentralized optimal tracking control for multi-agent system with large population and input constraint has been investigated. Conventional multi-agent optimal tracking control algorithms are facing difficulties caused by both communication and computation limits especially when the total number of agents goes to infinity, which is known as the Curse of Dimensionality. Moreover, the effects from practical scenario such as system input constraints has not thoroughly studied in optimal tracking control problems for massive multi-agent systems. In the paper, the Mean-Field game theory (MFG) theory has been adopted to break the Curse of Dimensionality firstly. Then, a novel online reinforcement learning algorithm, named Actor-Critic-Mass (ACM), has been designed to estimate the optimal tracking control for massive multi-agent systems. ACM has three Neural Networks (NNs), i.e. actor, critic, and mass NN, that can online approximate the optimal control policy, cost function, and all agents’ state probability density function. The effects from input constraints are integrated into the optimal tracking control problem through introducing a modified cost function. The NNs weights are effectively tuned by the Mean-Field equations. Eventually, the effectiveness and efficiency of the proposed optimal control method has been demonstrated through numerical simulations.