Research on improvement of model-free average reward reinforcement learning and its simulation experiment

Abstract

Traditional reinforcement learning always emphasizes the independent learning of a single agent. In Multi-Agent System (MAS), considering the relationship between independent learning and group learning, this paper presents a hybrid algorithm based on average reward reinforcement learning. In learning process of the modified algorithm, it still pays attention to the independent learning. In order to select an action which can reflect the multi-agent environmental information, we add the observed information and the prediction of other agent's actions when the learning agent chooses his action according to the current environmental state. The advantage of this design is that not only the agent will learn the optimal policy through autonomous study, but also as one member of MAS, the learning process can be integrated into the whole multi-agent environment. Robocup simulation league (2D) is a typical multi-agent system. By applying the new method to the training of the player, we prove the feasibility and validity of this algorithm.