Reinforcement Learning Method for Autonomous UAVs Monitoring an Uncertain Target

Abstract

Autonomous unmanned aerial vehicles are able to sense their surrounding environments, and fly safely with little or no human intervention. Autonomous unmanned aerial vehicles are characterized by their ability to make decisions based on predicting future possible situations and learning from previous experiences. In this paper, we aim at developing algorithms that enable unmanned aerial vehicles to monitor and detect a dynamic uncertain target autonomously. This work considers a real monitoring system consists of a mission area, an autonomous unmanned aerial vehicle, a charging station, and a dynamic uncertain target. The mission area consists of two main areas, which are the area where the charging station is placed and the area where the target moves. The target area is divided to a number of subareas. We also adopt a time slotted system that has M equal-duration slots. The unmanned aerial vehicle is equipped with a battery of finite energy that can be recharged from the charging station. It can fly from one subarea to another during one time slot. The target moves from one subarea to another according to an unknown Markov process. In this context, we propose to using reinforcement learning algorithms that enables autonomous unmanned aerial vehicles to learn the movement of a dynamic uncertain target autonomously. Simulation results show that reinforcement learning algorithms outperform the performance of random and circular algorithms. <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">1</sup> <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">1</sup> This work was supported by the ASPIRE Award for Research Excellence Program 2020 (Abu Dhabi, UAE) under grant AARE20-161.