Aerial-DeepSearch: Distributed Multi-Agent Deep Reinforcement Learning for Search Missions

Abstract

Search and Rescue (SAR) is an important part of several applications of national and social interest. Existing solutions for search missions in both terrestrial and aerial domains are mostly limited to single agent and specific environments; however, search missions can significantly benefit from the use of multiple agents that can quickly adapt to new environments. In this paper, we propose a framework based on Multi-Agent Deep Reinforcement Learning (MADRL) that realizes the actor-critic framework in a distributed manner for coordinating multiple Unmanned Aerial Vehicles (UAVs) in the exploration of unknown regions. One of the original aspects of our work is that the actors represent simulated or actual UAVs exploring the environment in parallel instead of traditional computer threads. Also, we propose addition of Long Short Term Memory (LSTM) neural network layers to the actor and critic architectures to handle imperfect communication and partial observability scenarios. The proposed approach has been evaluated in a grid world and has been compared against other competing algorithms such as Multi-Agent Q-Learning, Multi-Agent Deep Q-Learning to show its advantages. More generally, our approach could be extended to image-based/continuous action space environments as well.