A Reinforcement Learning Approach to Streetcar Bunching Control

Abstract

Transit agencies often implement operations control strategies so as to mitigate the effects of streetcar bunching along transit routes. Typical control strategies include vehicle holding, expressing, and short turning, which are usually implemented through manual means via field supervisors or central control centers. The objective of this study is to automate streetcar bunching control by means of multiple Reinforcement Learning (RL) agents that act on a series of successive signalized intersections. The multiple RL agents developed in this study include the “bunch–splitting,” “holding,” and “expressing” agents which work cooperatively to break up a streetcar bunch if one is detected and to build a reasonable headway between the paired streetcars. Various elements of the RL agents such as the action set, the reward, and the state space are set up under a traffic signal control context. Simulation results indicated that these multiple RL agents could split up a streetcar bunch and prevent it from forming again with a high success rate.