Multi-Agent Reinforcement Learning-Based Coordinated Dynamic Task Allocation for Heterogenous UAVs

Abstract

The coordinated dynamic task allocation (CDTA) problem for heterogeneous unmanned aerial vehicles (UAVs) in the presence of environment uncertainty is studied in this paper. Dynamic task allocation mainly solves the problem of resource reallocation after new tasks appear, so that the multi-UAV systems can quickly respond to further information and objectives. In this paper, the CDTA strategy for heterogenous UAVs is proposed through proposer-responser mechanism and prioritized experience replay, in which the multi-agent reinforcement learning (MARL)-based coordinated network is constructed to propose request, and the Q-network is developed to approximate expected return to determine the responser whether to participate in the dynamic task. The CDTA algorithm considers the uncertainty of dynamic task and has a high scalability in different UAV groups, which can reduce the burden of online calculation and increase the speed of online operation effectively. The experiment proves that the priority experience replay speeds up the convergence of the algorithm, and the scalability of the algorithm is verified within 10-180 UAVs. Comparison simulations with the game theory-based and reinforcement learning-based methods are provided to show the effectiveness of the proposed algorithm.