A Hierarchical Framework for Passenger Inflow Control in Metro System With Reinforcement Learning

Abstract

Since mobility needs grow rapidly, the metro system of modern cities is suffering from the oversaturated situation in the rush hour, which makes the metro system vulnerable and inefficient. Thus, passenger inflow control is proposed and implemented. This paper investigates the station-based passenger inflow control problem with the objective of maximizing overall transport efficiency and fairness. To solve the formulated nonlinear and nonconvex programming model, a novel framework integrating unsupervised subgoal discovery and hierarchical reinforcement learning, named SD-HIC, is proposed. With a series of subgoals, the complicated and long-horizon original task is decomposed and can be solved by reinforcement learning responsively and far-sightedly. A real-world case with operational data in the Guangzhou metro is presented to demonstrate the performance of the proposed model and framework. According to the results, the overall transport utility is improved by <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$28.87\%$</tex-math> </inline-formula> compared with the benchmark inflow control strategy that is frequently used in daily operations. Through solution algorithm studies and critical parameter analyses, the performance of the proposed passenger inflow control framework is further verified. Notably, the revealed subgoals are also distinguishable and interpretable, which is helpful for the operation staff in practice.