Adaptive Group-based Signal Control by Reinforcement Learning

Abstract

Group-based signal control is one of the most prevalent control schemes in the European countries. The major advantage of group-based control is its capability in providing flexible phase structures. The current group-based control systems are usually implemented with rather simple timing logics, e.g. vehicle actuated logic. However, such a timing logic is not sufficient to respond to the traffic environment whose inputs, i.e. traffic demands, dynamically change over time. Therefore, the primary objective of this paper is to formulate the existing group-based signal controller as a multi-agent system. The proposed signal control system is capable of making intelligent timing decisions by utilizing machine learning techniques. In this regard, reinforcement learning is a potential solution because of its self-learning properties in a dynamic environment. This paper, thus, proposes an adaptive signal control system, enabled by a reinforcement learning algorithm, in the context of group-based phasing technique. Two different learning algorithms, Q-learning and SARSA, have been investigated and tested on a four-legged intersection. The experiments are carried out by means of an open-source traffic simulation tool, SUMO. Performances on traffic mobility of the adaptive group- based signal control systems are compared against those of a well-established group-based fixed time control system. In the testbed experiments, simulation results reveal that the learning-based adaptive signal controller outperforms group-based fixed time signal controller with regards to the improvements in traffic mobility efficiency. In addition, SARSA learning is a more suitable implementation for the proposed adaptive group-based signal control system compared to the Q-learning approach.