Adaptive Traffic Signal Control Using Distributed MARL and Federated Learning

Abstract

Facing the increasingly serious traffic congestion problem, the existing traffic signal control technology has been unable to meet the demands of urban smart traffic construction. As one of the most promising methods, Reinforcement Learning (RL) has been widely exploited for solving the adaptive traffic signal control (ATSC) problem. However, centralized RL control mode is impractical due to its poor robustness and high computational complexity. In this paper, we propose two distributed Multi-agent Reinforcement Learning (MARL) control modes as well as a Federated Learning (FL) framework to solve the ATSC problem, where the former is based on Advantage Actor-Critic (A2C) algorithm and the latter is based on Federated Averaging (FedAvg) algorithm. By comparing in a small grid of traffic network, the experimental results reveal that our proposed algorithms outperform the centralized solution in terms of both the optimality and the learning efficiency.