Learning-based decision making for safe and scalable autonomous separation assurance

Abstract

With growing air traffic density in the national airspace and the introduction of new airspace operations such as urban air mobility (UAM) and urban traffic management (UTM), today's airspace operations are reaching a new level of complexity. In traditional airspace, human air traffic controllers face increased workload as a result of growing air traffic, resulting in a limitation in airspace capacity. To overcome this human cognitive limitation, automation tools provide a way to increase the sector capacity above human cognitive limits while reducing the workload for air traffic controllers, leading to a safer airspace environment. For the envisioned high-density airspace operations to become a reality, a scalable autonomous air traffic control system is required to comprehend the complex airspace, communicate with autonomous aircraft, and perform tactical decision making under uncertainty. In addition, the autonomous air traffic control system should be flexible enough to accommodate both traditional and new airspace operations. Separation assurance and conflict resolution is a key component of air traffic control (ATC). This task involves ensuring aircraft maintain safe separation requirements while also meeting required time of arrival (RTA) constraints at airspace metering fixes. In addition, when a potential loss of separation event is predicted, tactical maneuver advisories need to be prescribed by the ATC to the aircraft to resolve conflicts. In this dissertation, a suite of learning-based frameworks, as well as a new concept of operations (ConOps) of decentralized autonomous separation assurance and conflict resolution are introduced to accommodate the high-density, stochastic, and dynamic structured airspace that are flexible enough to be extended to low-altitude airspace operations. First, a new ConOps for separation assurance is proposed where the task is shifted from a centralized human ATC to a decentralized framework where each aircraft is equipped with autonomous self-separation. This allows for framework scalability that is invariant to the number of aircraft in the airspace. Second, a key component to enable autonomous separation assurance is data scalabilty and generalization under uncertainty. In this dissertation, the separation assurance task is formulated as a Markov Decision Process (MDP) and solved using deep multi-agent reinforcement learning. For high-density airspace environments, a novel intruder aircraft encoding technique is introduced that leverages attention networks to achieve data scalibility, without sacrificing important information from the environment. In addition, to make the framework more practical, we demonstrate how the framework is able to handle various levels of communication uncertainty in the environment. In this case,