Feasibility of reinforcement learning for UAV-based target searching in a simulated communication denied environment

Abstract

Target searching is crucial in real-world scenarios such as search and rescue in disaster sites and battlefield target reconnaissance. Unmanned aerial vehicles (UAVs) are an ideal technical solution for target searching in large-scale and high-risk areas because they are agile, low cost, and able to collaborate and carry different sensors. In complex scenarios like battlefields, due to the lack of communication infrastructures and the intensive interference, UAVs often operate in communication denied environments. As a result, fast and reliable communication channels between UAVs and ground operators are difficult to establish. Thus, in such conditions, UAVs must be able to complete tasks autonomously and intelligently, without receiving real-time commands from the operators. With the rapid advances in artificial intelligence, reinforcement learning has shown potentiality for solving continuous decision problems. The target searching problem studied in this paper falls into this category and is suitable for adopting reinforcement learning technologies. However, the feasibility of reinforcement learning in UAV-based target searching in communication denied environments is not clear and, thus, requires in-depth investigations. As a pilot study in this direction, this paper models the target searching problem in communication denied and confrontation situations and proposes a simulation environment based on this model. Extensive experiments are conducted to answer the following questions. (1) Can reinforcement learning be applied in target searching by multi-UAVs in communication denied environments? (2) What are the advantages and disadvantages of different reinforcement learning algorithms in solving this problem? (3) How the degree of communication denial influences the performance of these algorithms? The current mainstream reinforcement learning technologies are adopted to perform simulations, whose results are analyzed quantitatively, leading to the following observations. (1) Reinforcement learning can effectively solve target searching problems for multi-UAVs in communication denied environments. (2) Compared with other algorithms, an autonomous decision-making UAV cluster based on a deep Q-network (DQN)exhibits the best problem-solving ability. (3) The algorithm performance changes with the degree of communication denial but remains largely stable when the communication condition varies.