Decentralized Adaptive Optimal Control for Massive Multi-agent Systems Using Mean Field Game with Self-Organizing Neural Networks

Abstract

In this paper, a decentralized adaptive optimal control based on the Mean Field game and self-organizing neural networks has been developed for multi-agent systems (MAS) with a large population and uncertain dynamics. This design can effectively break "Curse of dimensionality" as well as reduce the computational complexity through appropriately integrating emerging mean-field game theory with self-organizing neural networks based reinforcement learning technique. Firstly, decentralized optimal control for massive multi-agent systems has been formulated as a mean-field game. To obtain the mean-field game solution, coupled Hamiltonian-Jacobian-Bellman (HJB) equation and Fokker-Planck-Kolmogorov (FPK) equation needed to be solved simultaneously which is challenging in real-time. Therefore, a novel Actor-Critic-Mass (ACM) structure has been developed along with self-organizing neural networks. In the developed ACM structure, each agent has three neural networks (NN), which is, 1) mass NN that learns the team's overall behavior via online estimating the solution of Fokker-Planck-Kolmogorov (FPK) equation, 2) critic NN that obtains the optimal cost function via learning the Hamiltonian-Jacobian-Bellman (HJB) equation solution along with time. 3) actor NN that estimates the decentralized optimal control by using the critic and mass NNs. To reduce the NNs computational complexity, a self-organizing neural network has been adopted that can adjusting NNs' architecture based on the NN learning performance as well as the computation cost. Eventually, numerical simulation has been provided to demonstrate the effectiveness of the developed scheme.