On Fast Learning of Cooperative Transport by Multi-robots using DeepDyna-Q

Abstract

In recent years, there have been many studies on cooperative transport. Reinforcement learning has attracted attention for formation changes. As reinforcement learning requires a large number of learning cycles, fast learning algorithms are needed. Model-based learning has been studied for unknown environments to learn efficiently. Deep Dyna-Q, a model-based learning method, has been proposed. In this paper, reinforcement learning using model-based learning is applied to formation changes of cooperative transportations. This study constructed Deep Dyna-Q using model-based learning to improve learning speed and applied it to robot formation change. The results showed that Deep Dyna-Q improved the number of episodes by about half compared to DQN. The formation change achievement rate in Deep Dyna-Q was 74% in an environment with a transport object and was slightly low. As it was reported that the achievement rate of the literature was 99%, an improvement in the success rate is desired. Future studies should aim to use a multi-agent algorithm MADDPG that uses model-based learning to achieve both achievement rate and learning time.