Reinforcement learning and aggregation

Abstract

Reinforcement learning (RL) is a learning technique that provides a means for learning an optimal control policy when the dynamics of the environment under consideration is unavailable [L.P. Kaelbling et al., 1996, R.S. Sutton and A.G. Barto, 1998]. While RL has been successfully applied in many single or multiple agents systems [S. Arai et al., 2000, H.R. Berenji and D.A. Vengerov, 2000, M. Tan, 1993, Y. Nagayuki et al., 2000], the learning quality is greatly influenced by learning algorithms and their parameters. Setting of the parameters of RL algorithms is something of a black art, and small differences in these parameters can lead to large differences in learning qualities. Determining the best algorithm and the optimal parameters can be costly in terms of time and computation. Even if the cost is acceptable, the robustness of learning is still a question. In order to address the difficulty, an aggregated multiagent reinforcement learning system (AMRLS) is proposed to deal with the RL environment as a multiagent environment. A maze world environment is used to validate the AMRLS. Experimental results illustrate that compared with normal Q(/spl lambda/)-learning and SARSA(/spl lambda/) algorithms, the AMRLS increases both the learning speed and the rate of reaching the shortest path.