A Multi-Agent Reinforcement Learning Approach to Traffic Control at Future Urban Air Mobility Intersections

Abstract

Today, air traffic controllers communicate with pilots via radio to direct the aircraft. The increasing demand in drone delivery and air mobility will increase air density, which requires highly automated air traffic control systems. To meet the growing demand in air transportation, a high standard autonomous support system is needed. In particular, we need an autonomous air traffic controller to keep the airspace safe and efficient. In this study, we propose a novel Multi-Agent Reinforcement Learning (MARL) approach to handle high-density UAM operations by providing effective guidance to electric vertical takeoff and landing (eVTOL) vehicles to avoid traffic congestion and reduce travel time. The goal of our MARL approach is to reduce the time at the environed urban air intersections by providing the speed advisories to each approaching vehicle for safe separation at the intersection. The proposed model is trained and evaluated in BlueSky, an open-source air traffic control simulation environment. The results of our simulations with real-world data from thousands of aircraft show that using MARL for the separation problem at the intersection is very promising for solving the problem of en-route air traffic control.