Designing Traffic Management Strategies Using Reinforcement Learning Techniques

Abstract

The future vision for traffic flow management is one that leverages advanced automation to assist human decision-makers in the identification of potential constraints and the development of resolution strategies. What makes this problem so challenging is the inherent uncertainty associated with forecasting these constraints, leaving human decision-makers reliant on experience to devise effective traffic management initiatives to mitigate demand in excess of resource capacity. This paper proposes to employ artificial intelligence-based methods to recommend traffic management initiatives under forecast uncertainty and to do so in a real-time planning context. The proposed algorithm consists of 1) a policy network that is generated offline using an Expert Iteration algorithm, 2) a statistical model that updates the likelihood of constraint futures based on observations, and 3) a Monte Carlo tree search algorithm that explores possible combinations of traffic management initiatives to identify the recommended actions for the current decision. The skill introduced by each of the algorithmic components is assessed for a case study focused on managing arrivals into the Atlanta Hartsfield–Jackson International Airport over 92 validation days.