Efficient Traffic Signal Control Research Articles

Reinforcement Learning for Transit Signal Priority with Priority Factor

Public transportation has been identified as a viable solution to mitigate traffic congestion. Transit signal priority (TSP) control, which is widely used at signalized intersections, has been recognized as a practical strategy to improve the efficiency and reliability of bus operations. However, traditional TSP control may fall short of efficiency and is facing several challenges of negative externalities for non-transit users and the need to handle conflicting priority requests. Recent studies have proposed the use of reinforcement learning (RL) methods to identify efficient traffic signal control (TSC). Some of these studies on RL-based TSC have incorporated the concept of max-pressure (MP), which is a maximal weight-matching algorithm to minimize queue sizes. Nevertheless, the existing RL-based TSC methods focus on private vehicles and cannot adequately distinguish between buses and private vehicles. In prior research, RL-based control has been implemented within the context of bus rapid transit (BRT) systems. This study proposes a novel RL-based TSC strategy that leverages the MP concept and extends it to incorporate TSP control. This is the first implementation of RL-based TSP control within the mixed-traffic road network. A significant innovation of this research is the introduction of the priority factor (PF), which is designed to prioritize bus movements at signalized intersections. The proposed RL-based TSP with PF control seeks to balance the competing objectives of enhancing bus operations while mitigating adverse impacts on non-transit users. To evaluate the performance of the proposed TSP method with the PF mechanism, simulations were conducted on an arterial and a grid network under dynamic traffic conditions. The simulation results demonstrated that the proposed TSP with PF not only reduces bus travel times and resolves conflicts between priority requests but also does not make a significant negative impact on passenger car operations. Furthermore, the PF can be dynamically assigned according to the number of passengers on each bus, suggesting the potential for the proposed approach to be applied in various traffic management scenarios.

OAM: An Option-Action Reinforcement Learning Framework for Universal Multi-Intersection Control

Efficient traffic signal control is an important means to alleviate urban traffic congestion. Reinforcement learning (RL) has shown great potentials in devising optimal signal plans that can adapt to dynamic traffic congestion. However, several challenges still need to be overcome. Firstly, a paradigm of state, action, and reward design is needed, especially for an optimality-guaranteed reward function. Secondly, the generalization of the RL algorithms is hindered by the varied topologies and physical properties of intersections. Lastly, enhancing the cooperation between intersections is needed for large network applications. To address these issues, the Option-Action RL framework for universal Multi-intersection control (OAM) is proposed. Based on the well-known cell transmission model, we first define a lane-cell-level state to better model the traffic flow propagation. Based on this physical queuing dynamics, we propose a regularized delay as the reward to facilitate temporal credit assignment while maintaining the equivalence with minimizing the average travel time. We then recapitulate the phase actions as the constrained combinations of lane options and design a universal neural network structure to realize model generalization to any intersection with any phase definition. The multiple-intersection cooperation is then rigorously discussed using the potential game theory. We test the OAM algorithm under four networks with different settings, including a city-level scenario with 2,048 intersections using synthetic and real-world datasets. The results show that the OAM can outperform the state-of-the-art controllers in reducing the average travel time.

Deep learning and case-based reasoning for predictive and adaptive traffic emergency management

An efficient traffic signal control system (TSCS) should not only be reactive to the current traffic but also be predictive by anticipating future traffic disturbances. In this study, we investigate the potential of using convolution neural network (CNN) in detecting emergency cases and forecasting events that can interrupt the traffic flow. Case-based reasoning (CBR) is then exploited to react to detected and forecasted events. We further develop an adapted Reinforcement Leaning (RL) algorithm in building and enhancing the case bases. The proposed system inherits the advantages of CNN, CBR, and RL, which allow detection, prediction, control, evaluation, and learning in a unified framework. To assess the proposed TSCS, we compare our approach with a set of state-of-art algorithms (e.g., multi-agent preemptive case-based reasoning algorithm and multi-agent preemptive longest queue first—maximal weight matching). The proposed TSCS outperforms the benchmarking algorithms through experiments in various traffic scenarios.

The Real-time Traffic Signal Control System for the Minimum Emission using Reinforcement Learning in V2X Environment

As the population and vehicle ownership increase, emission of pollutants is also increasing. The percentage of GHG emission by transportation sector is about 21 % in 2015 (OECD), and this may be caused by frequent stop-and-go phenomenon or delay time of vehicles in signalized intersection. Generally, these could be minimised by driving in constant speed or decreasing the delay times with an efficient traffic signal control. On the other hand, researches try to decrease vehicles’ delay time and to exclude the unnecessary stop-and-go phenomenon in an urban signalized intersection with an advent of V2X (Vehicle-to-Everything) technology development. Especially, in traditional pre-timed traffic signal control situation, even the autonomous vehicles would be impossible to exhibit their own maximum performance. Thus, the development of the traffic signal control system could have effects not only on the traffic flow but also on environmental aspects, which optimizes the signalized traffic flow based on the real-time vehicle information. In this research, on the premise of V2X environment, changes in traffic flow and the emission are analysed based on microscopic traffic information. In specific, the reinforcement learning model is constructed based on Deep Learning which learns the real-time traffic information and displays the optimal traffic signal. The performance of the system was analysed through microscopic traffic simulator - Vissim. The proposed system is expected to contribute on analysing the traffic flow and the environmental effects. Also, it is expected to contribute on constructing the green smart cities with an advent of autonomous vehicle operation in future V2X environment.

Reinforcement learning-based multi-agent system for network traffic signal control

A challenging application of artificial intelligence systems involves the scheduling of traffic signals in multi-intersection vehicular networks. This paper introduces a novel use of a multi-agent system and reinforcement learning (RL) framework to obtain an efficient traffic signal control policy. The latter is aimed at minimising the average delay, congestion and likelihood of intersection cross-blocking. A five-intersection traffic network has been studied in which each intersection is governed by an autonomous intelligent agent. Two types of agents, a central agent and an outbound agent, were employed. The outbound agents schedule traffic signals by following the longest-queue-first (LQF) algorithm, which has been proved to guarantee stability and fairness, and collaborate with the central agent by providing it local traffic statistics. The central agent learns a value function driven by its local and neighbours' traffic conditions. The novel methodology proposed here utilises the Q-Learning algorithm with a feedforward neural network for value function approximation. Experimental results clearly demonstrate the advantages of multi-agent RL-based control over LQF governed isolated single-intersection control, thus paving the way for efficient distributed traffic signal control in complex settings.