Supervised deep actor network for imitation learning in a ground-air UAV-UGVs coordination task

Abstract

Ground-Air coordination is a very complex environment for a machine learning algorithm. We focus on the case where an Unmanned Aerial Vehicle (UAV) needs to support a group of Unmanned Ground Vehicles (UGVs). The UAV is required to broadcast an image that contains all UGVs, thus, offering a bird-eye-view on the group as a whole. The source of complexity in this task is twofold. First, coordination needs to occur without communication between the UAV and UGVs. Second, the ability of the UAV to sense the UGVs is coupled with the ability of the UAV to learn how to track laterally the UGVs and adapt its vertical position so that the images of the UGVs are appropriately spaced within the camera field of view. In this paper, we propose using the Deep Actor Network component of an Actor-Critic Deep Reinforcement Learning architecture as a supervised learner. The advantage of this approach is that it offers a step towards autonomous learning whereby the full Actor-Critic model can be utilized in the future. Human demonstrations are collected for the deep Actor network to learn from. The system is built using the Gazebo Simulator, Robot Operating System, and the OpenAI Gym. We show that the proposed setup is able to train the UAV to follow the UGVs while maintaining all UGVs within camera range in situations where UGVs are performing complex maneuvers.