MLDRL: A multi-layer distributed reinforcement learning framework with multiple trainers

Abstract

Synchronous distributed sampling is widely used in various distributed reinforcement learning algorithms. However, it is difficult to scale to a large case due to significant synchronization and communication costs. In this paper, we propose a scalable layered distributed reinforcement learning framework by introducing multiple trainers. We use a parameter processor to coordinate parameters obtained from multiple trainers and introduce worker nodes to reduce the communication cost among the parameter processor and the trainers. We do experiments in a multi-agent environment, and the results show that the proposed framework significantly accelerates the training efficiency of the reinforcement learning algorithm.