Large-Scale Traffic Grid Signal Control Using Decentralized Fuzzy Reinforcement Learning

Abstract

With the rise of rapid urbanization around the world, a majority of countries have experienced a significant increase in traffic congestion. The negative impacts of this change have resulted in a number of serious and adverse effects, not only regarding the quality of daily life at an individual level but also for nations’ economic growth. Thus, the importance of traffic congestion management is well recognized. Adaptive real-time traffic signal control is effective for traffic congestion management. In particular, adaptive control with reinforcement learning (RL) is a promising technique that has recently been introduced in the field to better manage traffic congestion. Traditionally, most studies on traffic signal control have used centralized reinforcement learning, whose computation inefficiency prevents it from being employed for large traffic networks. In this paper, we propose a computationally cost-effective distributed algorithm, namely, a decentralized fuzzy reinforcement learning approach, to deal with problems related to the exponentially growing number of possible states and actions in RL models for a large-scale traffic signal control network. More specifically, the traffic density at each intersection is first mapped to four different fuzzy sets (i.e., low, medium, high, and extremely high). Next, two different kinds of algorithms, greedy and neighborhood approximate Q-learning (NAQL), are adaptively selected, based on the real-time, fuzzified congestion levels. To further reduce computational costs and the number of state-action pairs in the RL model, coordination and communication between the intersections are confined within a single neighborhood, i.e., the controlled intersection with its immediate neighbor intersections, for the NAQL algorithm. Finally, we conduct several numerical experiments to verify the efficiency and effectiveness of our approach. The results demonstrate that the decentralized fuzzy reinforcement learning algorithm achieves comparable results when measured against traditional heuristic-based algorithms. In addition, the decentralized fuzzy RL algorithm generates more adaptive control rules for the underlying dynamics of large-scale traffic networks. Thus, the proposed approach sheds new light on how to provide further improvements to a networked traffic signal control system for real-time traffic congestion.