With the continuous advancement of Unmanned Aerial Vehicle (UAV) swarm technology, the field of adaptive decision-making for heterogeneous UAV swarms across a spectrum of tasks is rapidly emerging as a focal point of research. This domain holds significant importance for enhancing the operational efficiency and responsiveness of UAV swarms in complex and dynamic environments. This paper addresses this challenge by introducing an innovative adaptive task decision-making method, which leverages Deep Q-Network (DQN) algorithms, specifically tailored to meet the unique requirements of heterogeneous UAV swarms. We initially construct a mathematical model that transforms the intricate swarm decision-making issues into a standard Markov Decision Process (MDP), and meticulously design the state space, action space, reward function, and state transition function to cater to the decision-making needs of UAV swarms. Utilizing this model, we further integrate a hybrid experience replay mechanism, coupled with DQN algorithms for offline training, enabling the efficient transformation of training outcomes into a network model that is directly applicable to real-time online decision-making. This approach markedly enhances the decision-making adaptability and dynamic adaptability of UAV swarms when confronted with a variety of tasks. During the offline training phase, we also employ a hybrid experience replay mechanism and target network approximation techniques to bolster the network's stability and training efficiency, ensuring high reliability in online decision-making. Comparative simulations with existing DQN algorithms reveal the distinct superiority of our method in addressing adaptive task decision-making challenges. Moreover, through 1000 comprehensive simulation experiments, we further validate the stability and effectiveness of our method across diverse task environments, providing a solid theoretical foundation and technical support for the practical application and proliferation of UAV swarm technology.
Read full abstract