Multi-objective multi-agent deep reinforcement learning to reduce bus bunching for multiline services with a shared corridor

Abstract

Bus bunching is a long-standing problem in transit operation and ruining the regularity of transit service. In a typical urban transit network setting of multiple lines with a shared corridor, bus bunching becomes more frequent as there is more uncertainty inside the shared corridor. While multi-agent reinforcement learning (MARL) has been a promising scheme for learning efficient control policy in a multi-agent system, few studies have explored its applicability in multi-line transit control scenarios. In this study, we focus on a basic transit network where there are two bus lines with a shared corridor. An efficient MARL framework is proposed to learn multi-line bus holding control to avoid bus bunching. Specifically, we design observation and reward functions that incorporate multi-line information. In addition, a preference weights producer is introduced to update the objective weights towards a good trajectory evaluation during daily transit operation. In this way, we handle the multi-objective issue in multi-line control. In experimental studies, we validate the superiority of the method in real-world bus lines. Results show that the state and reward augmented with multi-line information benefit MARL in multi-line bus control. Besides, by updating preference weights towards less passenger waiting time, the regularity of transit service is further improved.