Cooperative Traffic Signal Control based on Biased ReLU Neural Network Approximation

Abstract

Traffic signal control is important in intelligent transportation system, of which the cooperative control is difficult to realize but yet vital. Popular methods for solving this problem are based on multi-agent reinforcement learning (RL), in which function approximator, e.g., different kinds of neural network play a critical role. In this paper, we propose a multi-agent actor-critic RL framework with global value function and local policy function, for which the piecewise linear neural network, named biased ReLU (BReLU) is used as the function approximator. The reason for doing this is two-fold. First, it has been proved in the control literature that minimizing (maximizing) a piecewise linear function over a polyhedron yields piecewise linear solutions. Second, the BReLU neural network can provide a more accurate approximation than the traditional ReLU neural network when they have similar network structures. The proposed method is evaluated on the Simulation of Urban Mobility (SUMO) environment compared with two benchmark traffic signal control methods. The simulated results illustrate the proposed algorithm can coordinate the signal control between different intersections, achieve lower and more sustainable intersection delays on the whole traffic network.