ATFM computational agent based on reinforcement learning aggregating human expert experience

Abstract

Air Traffic Management (ATM) is defined as a broad domain that consists of complex activities such as Air Traffic Flow Management (ATFM). Within this context this paper presents two prototypes of a module (MAAD) assembled in a decision support device that is based on Artificial Intelligence, which generates suggestions for air traffic flow control according to each traffic saturation scenario. The device known as Distributed Decision Support System for Air Traffic Flow Management (SISCONFLUX) was built with a computational agent based on Reinforcement Learning (Module MAAD), which uses Q-learning algorithm. Two prototypes were built and one of them incorporated the specialist's (human agent) experience. The performance of the prototypes was evaluated through a case study modeled from real traffic demands verified on certain dates. The results obtained in the case study are promising; the behavior of the prototypes demonstrated that the Q-learning algorithm converged in a satisfying way. The prototypes generated actions that contributed effectively to the reduction of saturation in the air traffic scenarios under test.