Maneuver Decision of UAV in Short-Range Air Combat Based on Deep Reinforcement Learning

Qiming Yang,Guoqing Shi,Jiandong Zhang,Jinwen Hu,Yong Wu

doi:10.1109/access.2019.2961426

Qiming Yang, Guoqing Shi + Show 3 more

Open Access

PDF Available

https://doi.org/10.1109/access.2019.2961426

Copy DOI

Export

Save

Cite

Journal: IEEE Access	Publication Date: Jan 1, 2020
Citations: 117	License type: CC BY 4.0

Affiliation: Northwestern Polytechnical University

Abstract
Highlights/Summary
Full-Text PDF
Similar Papers

Abstract

Listen

With the development of artificial intelligence and integrated sensor technologies, unmanned aerial vehicles (UAVs) are more and more applied in the air combats. A bottleneck that constrains the capability of UAVs against manned vehicles is the autonomous maneuver decision, which is a very challenging problem in the short-range air combat undergoing highly dynamic and uncertain maneuvers of enemies. In this paper, an autonomous maneuver decision model is proposed for the UAV short-range air combat based on reinforcement learning, which mainly includes the aircraft motion model, one-to-one short-range air combat evaluation model and the maneuver decision model based on deep Q network (DQN). However, such model includes a high dimensional state and action space which requires huge computation load for DQN training using traditional methods. Then, a phased training method, called “basic-confrontation”, which is based on the idea that human beings gradually learn from simple to complex is proposed to help reduce the training time while getting suboptimal but efficient results. Finally, one-to-one short-range air combats are simulated under different target maneuver policies. Simulation results show that the proposed maneuver decision model and training method can help the UAV achieve autonomous decision in the air combats and obtain an effective decision policy to defeat the opponent.

Highlights

INTRODUCTIONMilitary unmanned aerial vehicles (UAVs) have attracted much attention for their low cost, long flight time and fearless sacrifice
Compared with manned aircraft, military unmanned aerial vehicles (UAVs) have attracted much attention for their low cost, long flight time and fearless sacrifice
Aiming at the problem of low learning efficiency and local optimization due to the large state space of air combat, this paper proposed a model training method based on the principle of going to confrontation training from basic training

Summary

INTRODUCTION

Military UAVs have attracted much attention for their low cost, long flight time and fearless sacrifice. Autonomous maneuver decision requires automatically generating flight control commands under various air combat situations based on technical methods such as mathematical optimization and artificial intelligence. By solving the optimization model, the maneuver policy can be obtained Many of these optimization algorithms have poor real-time performance on large-scale problems and cannot implement online decision making for air combat. In the paper, the UAV is assumed to move in a 2D plane, and the actual situation of the aircraft moving in 3D space in air combat is not considered. In this paper, based on reinforcement learning, the UAV short-range air combat autonomous maneuver decision modeling is carried out. The research focus of this paper is maneuvering decision-making, which mainly considers the positional relationship and velocity vector of the two sides in the three-dimensional space.

ONE-TO-ONE SHORT-RANGE AIR COMBAT EVALUATION MODEL

MANEUVER DECISION MODELING BY DEEP Q NETWORK

TRAINING EVALUATION

CONCLUSION