Digital-Twin-Assisted Task Assignment in Multi-UAV Systems: A Deep Reinforcement Learning Approach

Abstract

Most existing multi-unmanned aerial vehicle (multi-UAV) systems focus on fly path or energy consumption for task assignment, while little attention has been paid to the dynamic feature of the task, resulting in poor task completion ratio. The machine learning (ML) paradigm provides new methodologies for task assignment. However, ML methods are usually of heavy resource-consumption that cannot be directly applied in the UAV. In this paper, a digital twin (DT) assisted task assignment approach is proposed to improve the resource-intensive utilization and the efficiency of deep reinforcement learning (DRL) in multi-UAV system. The approach has a three-layer network structure which can dynamically assign tasks based on the task time constraints. Moreover, the approach is divided into two stages of initial task-assignment and task-reassignment. In the first stage, airship divides a task into multiple subtasks according to the shortest distance based on genetic algorithm and assigns them to UAVs. In the second stage, the DT can be leveraged to enable the airships to learn from the features of tasks and to generate the Q-value of the estimated value network of DRL for UAVs via pre-train of DT. The Q-value can be directly applied for deep Q-learning network (DQN) in the UAVs to reduce the training episode. Furthermore, the DQN is adopted to train task-reassignment strategy. Simulation results indicate that the DQN with DT can significantly reduce the training episode, improving 30% of the task completion ratio and 19% of the system energy efficiency compared with that of the baseline methods.