Studies on Hierarchical Reinforcement Learning in Multi-Agent Environment

Abstract

Reinforcement learning addresses the problem of learning to select actions in order to maximize an agent's performance in unknown environments. To scale reinforcement learning to complex real-world tasks, agent must be able to discover hierarchical structures within their learning and control systems. In this paper, the use of hierarchical reinforcement learning (HRL) to speed up the acquisition of cooperative multi-agent tasks is investigated, and a hierarchical multi-agent reinforcement learning (RL) framework and a hierarchical multi-agent RL algorithm called cooperative HRL are proposed. A fundamental property of the proposed approach is that it allows agents to learn coordination faster by sharing information at the level of cooperative subtasks, rather than attempting to learn coordination at the level of primitive actions. This approach can significantly speed up learning and make it more scalable with the number of agents.