An apollonius circle based game theory and Q-learning for cooperative hunting in unmanned aerial vehicle cluster

Abstract

Cooperative hunting has attracted great research interests with both pursuer and evader behavior strategies. Existing approaches typically utilize computing power to improve the accuracy of hunting. However, these methods ignore the real-time characteristic of unmanned aerial vehicle (UAV) cluster and timeliness of hunting process, directly using them into UAV cluster application would lose efficacy. To solve the problem of cooperative hunting of UAV cluster (pursuers) for one intelligent UAV (evader), we propose an apollonius circle based game theory and Q-learning for cooperative hunting (ACGQ-CH). Specifically, it constructs the behavior strategies and payment matrix of the pursuers and the evader by using game theory and apollonius circle. Then, a state-action matrix is built and a dynamically adjusting the greedy factor is designed based on Q-learning algorithm and reward mean, respectively. Finally, it derives Nash equilibrium solution by solving the state-action matrix, and guides the pursuers to achieve cooperative hunting. The simulation results demonstrate our approach reduces the number of learning steps by 50.8% compared to traditional Q-learning and reduces the hunting time by 16.83, 27.35 and 12.56% respectively compared to baseline methods.