Co-operative Multi-agent Twin Delayed DDPG for Robust Phase Duration Optimization of Large Road Networks

Abstract

AbstractLarge road networks overflowing with vehicles have called for increased traffic congestion, the impact of which is felt on an everyday basis and across different dimensions like decreased traveller satisfaction, increased fuel usage and increased air pollution among many other troubles. Improved traffic control strategies that can self-learn to adapt their decisioning in response to dynamic changes in the traffic flows and are capable of mitigating overall network congestion as opposed to localized congestion at intersections, are of great importance in mitigating traffic congestion. Traffic control strategies which were rule-based or historical-demand based were over-simplified and could not scale to large real-world road networks. To effectively control traffic congestion at scale, the need for co-operation and communication between the different intersections of a large road network is crucial. Multi-agent reinforcement learning methods are an apt choice for traffic signal control of large scale road networks as they can learn to perform predictive control actions that will reduce overall network congestion dynamically at scale. In this paper, we extend the work done in [24] to traffic signal timing (green phase duration) control using Multi-agent Twin Delayed Deep Deterministic Policy Gradients (MATD3) on large scale real-world road networks. The solution strategy was exposed to simulations of different road networks and time-varying traffic flows. The experimental results showed that our strategy is robust to the different kinds of road networks and vehicular traffic flows, and consistently outperformed its adaptive and rule-based counterparts by significantly reducing the average vehicular delay and queue length.KeywordsTraffic signal timing controlMulti-agent reinforcement learningDeep reinforcement learningAdaptive congestion control