Multi-agent deep reinforcement learning with actor-attention-critic for traffic light control

Abstract

In recent years, with the increase of urbanization and car ownership, urban traffic congestion have become increasingly prominent. Traffic light control can effectively reduce urban traffic congestion. In the research of controlling traffic lights of multiple intersections, most methods introduced theories related to deep reinforcement learning, but few methods considered the information interaction between intersections or the way of information interaction is unreasonable. Inspired by this, this paper proposes a multi-agent deep reinforcement learning with actor-attention-critic network for traffic light control (MAAC-TLC) algorithm. In MAAC-TLC, each agent introduces the attention mechanism in the process of learning, so that it will not pay attention to all the information of other agents indiscriminately, but only focus on the important information of the agents that plays an important role in it, so as to ensure that all intersections can learn the optimal policy. Finally, the traffic lights at each intersection in the MAAC-TLC algorithm are controlled according to its own policy, thereby improving the traffic efficiency of the traffic network. The experimental results on light and heavy traffic flow scenarios have demonstrated that MAAC-TLC can improve traffic congestion at multiple intersections effectively.