An inductive heterogeneous graph attention-based multi-agent deep graph infomax algorithm for adaptive traffic signal control

Abstract

Adaptive traffic signal control (ATSC) facilitates alleviating traffic congestion. Multi-agent deep reinforcement learning (MDRL) is a new promising algorithm for ATSC, and Graph Neural Networks (GNNs) further promote its learning ability. However, there are some drawbacks in the state-of-the-art MDRL algorithms. (1) These algorithms cannot effectively fuse diverse heterogeneous information of traffic networks due to adopting homogeneous GNNs; (2) These algorithms cannot be effectively trained due to merely adopting MDRL loss functions. In this paper, we propose an Inductive Heterogeneous graph Attention-based Multi-agent Deep Graph Infomax (IHA-MDGI) algorithm for ATSC. The proposed IHA-MDGI algorithm conducts both feature fusion via a proposed Inductive Heterogeneous graph Attention (IHA) algorithm and training via a proposed Multi-agent Deep Graph Infomax (MDGI) framework. Specifically, (1) Unlike the algorithms which adopt homogeneous GNNs, in the IHA algorithm, heterogeneous GNNs are designed to fuse both heterogeneous structural information and heterogeneous features of traffic networks, which aims to acquire heterogeneous information embeddings of traffic networks. (2) In the MDGI framework, the acquired embeddings are used to calculate the signal-control policies and Q-value for each agent, and then a mutual-information loss function is designed, which combines with the MDRL loss function to jointly train the whole algorithm. The designed mutual-information loss function focuses on maximizing mutual information between input (i.e., heterogeneous information embeddings) and output (i.e., Q-value), which can produce cooperative signal-control policies and maximize Q-value. We conduct the experiments in both real-traffic and synthetic-traffic networks under the time-varying traffic flows, and the results demonstrate that IHA-MDGI algorithm outperforms the state-of-the-art MDRL algorithms about multiple metrics.