An Efficient Aircraft Conflict Detection and Resolution Method Based on an Improved Reinforcement Learning Framework

Abstract

With the steady increase of air traffic column, an auxiliary decision tool is required to compensate the operation redundancy deficiency of more sectors of air traffic control. To solve the problem of nonconflict high-density departure and arrival traffic flow, this method is expected to rapidly establish and maintain safe separation with more flexible changing strategies for aircraft heading and speed. This paper proposes an improved reinforcement learning framework to achieve conflict detection and resolution. The proposed framework includes the first development of an air traffic flow model based on a multiagent Markov decision process. The goal reward function was then maximized by improved Monte-Carlo tree search combined with an upper confidence bound tree. Three simulation scenarios were designed for illustrating the improvements of the proposed algorithm, with the results indicating that the algorithm could establish and maintain safe separation between 20 agents in the simplified hexagon-shaped airspace of Huadong, China. Furthermore, the proposed method was demonstrated to reduce the number of conflicts between aircraft agents by up to 26.32% compared to previous research.