Decentralized Function Approximated Q-Learning in Multi-Robot Systems For Predator Avoidance

Abstract

The nature-inspired behavior of collective motion is found to be an optimal solution in swarming systems for predator avoidance and survival. In this work, we propose a two-level control architecture for multi-robot systems (MRS), which leverages the advantages of flocking control and function approximated reinforcement learning for predator avoidance task. Reinforcement learning in multi-agent systems has gained a tremendous amount of interest in recent few years. Computationally intensive architectures such as deep reinforcement learning and actor-critic approaches have been extensively developed and have proved to be extremely efficient. The proposed approach, comprising of cooperative function approximated Q-learning, is applied such that it ensures formation maintenance in MRS while predator avoidance. A consensus filter is incorporated into the control architecture, to sense predators in close vicinity in a distributed and cooperative fashion to ensure consensus on states among the robots in the system. The proposed approach is proved to be convergent and results in superior performance in unexplored states and reduced number of variables. Simulation results confirm the effectiveness of the proposed approach over existing methods. We expect that the proposed approach can be conveniently applied to many other areas with little modifications, such as fire-fighting robots, surveillance and patrolling robots.