Traffic Signal Control Policies Research Articles

With spatial queueing, the P0 responsive traffic signal control policy may fail to maximise network capacity even if queue storage capacities are very large

The local responsive traffic signal control policy P0 was designed to maximise network capacity under certain conditions and it has been shown, in Smith (1979a, b, 1980) and Smith et al. (2019a, 2022), that the P0 policy and related policies do indeed maximise the capacity of many steady state networks or quasi-dynamic networks with vertical and spatial queues under various conditions. This current paper shows, by giving an example, that if queueing is spatial then the original policy P0 itself may not maximise network capacity, even if the queue storage capacity of each link is very large.

Deep Reinforcement Learning for the Co-Optimization of Vehicular Flow Direction Design and Signal Control Policy for a Road Network

Reinforcement Learning (RL) is a popular approach for deciding on an optimum traffic signal control policy to alleviate congestion in a road network. However, the traffic signal control policy can also be optimized in conjunction with the design of vehicular flow directions to further improve traffic performance. The design of vehicular flow directions refers to the right of way or directional restriction imposed in a road network. Here, a new RL-based technique is presented for co-optimization of the design of vehicular flow directions and control policy for traffic signals. This technique consists of a two-step iterative process, wherein a set of vehicular flow directions for a road network is generated, then a RL-based approach is used to train the traffic signal control policy over the given set of vehicular flow directions. Following the proposed technique, the vehicular flow directions with poor traffic performance are iteratively eliminated, while new vehicular flow directions are generated to achieve better traffic performance and realize convergence to a maximum possible expected traffic performance. The proposed RL-based technique is evaluated by using two examples under rush hour and non-rush hour traffic conditions. It is found that, compared to a RL-based approach in which only traffic signal control policy is considered, the proposed approach can be used to obtain a better traffic performance in terms of vehicular queue length and throughput.

Optimizing Transportation Management with a Hybrid GNN-RL Approach

The process of rapid urbanization in contemporary cities has resulted in a rise in traffic congestion, accidents, and pollution, leading to negative impacts on transportation efficiency and urban development. In response, Intelligent Transportation Systems (ITS) have emerged as a crucial field of research and development, with a goal to leverage technological advancements to enhance transportation safety, efficiency, and sustainability. However, traditional expert systems and traffic control methods have displayed limitations in handling the complexities of urban transportation. To address these challenges in Advanced Traffic Management Systems (ATMSs), this paper pro- poses a new approach utilizing Graph Neural Network-Reinforcement Learning (GNN-RL). By using GNNs for pre-training and information aggregation from the entire grid, our method enables downstream RL tasks, allowing for fine-tuning of effective global information aggregation and dynamic traffic signal control policy adjustment. The GNN-RL approach offers enhanced adapt- ability and performance in managing long-term dependencies, rendering it a promising alternative for traffic control in contemporary urban transportation systems.

Back-Pressure traffic Signal Control in the Presence of Noisy Queue Information*

In this paper, we consider centralized traffic signal control policies using the max-weight algorithm when the queue size measurement is noisy. We first show analytically that the standard max-weight algorithm is throughput optimal even under noisy queue measurements. However, the average steady-state queue lengths and subsequently the average delays are increased. In order to alleviate the effect of these noisy measurements we add filtering to the max-weight algorithm; more specifically, we propose the Filtered-max-weight algorithm, which is based on particle filtering. We demonstrate via simulations that the Filtered-max-weight algorithm performs better than the standard max-weight algorithm in the presence of noisy measurements.

IPDALight: Intensity- and phase duration-aware traffic signal control based on Reinforcement Learning

Reinforcement Learning (RL) has been recognized as one of the most effective methods to optimize traffic signal control. However, due to the inappropriate design of RL elements (i.e., reward and state) for complex traffic dynamics, existing RL-based approaches suffer from slow convergence to optimal traffic signal plans. Meanwhile, to simplify the traffic modeling, most optimization methods assume that the phase duration of traffic signals is constant, which strongly limits the RL capability to search for traffic signal control policies with shorter average vehicle travel time and better GreenWave control. To address these issues, this paper proposes a novel intensity- and phase duration-aware RL-based method named IPDALight for the optimization of traffic signal control. Inspired by the Max Pressure (MP)-based traffic control strategy used in the transportation field, we introduce a new concept named intensity, which ensures that our reward design and state representation can accurately reflect the status of vehicles. By taking the coordination of neighboring intersections into account, our approach enables the fine-tuning of phase duration of traffic signals to adapt to dynamic traffic situations. Comprehensive experimental results on both synthetic and real-world traffic scenarios show that, compared with the state-of-the-art RL methods, IPDALight can not only achieve better average vehicle travel time and greenwave control for various multi-intersection scenarios, but also converge to optimal solutions much faster.

Double Deep Q-Network with a Dual-Agent for Traffic Signal Control

Adaptive traffic signal control (ATSC) based on deep reinforcement learning (DRL) has shown promising prospects to reduce traffic congestion. Most existing methods keeping traffic signal phases fixed adopt two agent actions to match a four-phase suffering unstable performance and undesirable operation in a four-phase signalized intersection. In this paper, a Double Deep Q-Network (DDQN) with a dual-agent algorithm is proposed to obtain a stable traffic signal control policy. Specifically, two agents are denoted by two different states and shift the control of green lights to make the phase sequence fixed and control process stable. State representations and reward functions are presented by improving the observability and reducing the leaning difficulty of two agents. To enhance the feasibility and reliability of two agents in the traffic control of the four-phase signalized intersection, a network structure incorporating DDQN is proposed to map states to rewards. Experiments under Simulation of Urban Mobility (SUMO) are carried out, and results show that the proposed traffic signal control algorithm is effective in improving traffic capacity.

A Game-Based Adaptive Traffic Signal Control Policy Using the Vehicle to Infrastructure (V2I)

In this paper, we propose a game-based policy for the signal control of an isolated intersection together with the rerouting of the vehicles in this intersection using V2I technology. The game starts at the beginning of each signal cycle on the condition that there are vehicles delayed in the intersection at the end of current cycle. Then the delayed vehicles negotiate with the signal controller to decide the signal setting for next cycle while rerouting their next movements. During the game, each delayed vehicle selects one route from their own acceptable route space to reduce the intersection delay time, while the signal controller adjusts the signal setting to meet the traffic demand of the delayed vehicles as well as maximize the throughput. The game can be terminated with end conditions in finite rounds. The vehicles coming after the end of game reroute in a similar way with the delayed vehicles but based on the game results. Numerical experiments on a calibrated network of Caohejing District in Shanghai indicate that our proposed method can effectively mitigate road congestions and improve the network capacity, especially at high traffic loads.

Traffic control which maximises network throughput: Some simple examples

This paper seeks traffic signal control policies which maximise long-run network throughput when demand is within network capacity and also when demand is beyond network capacity. The paper considers smooth versions of two well-known responsive traffic signal control policies: the P0 policy (see Smith, 1979b, 1980 or Smith et al., 2015) and the maxpressure or MP policy (see Varaiya, 2013a). The paper gives two simple example networks where both of these smooth responsive traffic control policies give rise to decreasing throughput for increasing demands; when these demands are beyond the network capacity. The paper then suggests a simple modification of the two responsive traffic control policies and shows that the two modified policies do maximise the long run throughput of the two signal-controlled networks; both when a steady input demand D is within network capacity and also when a steady demand exceeds network capacity. The modified version of P0 is as follows: for a certain positive number K,forD⩽Kuse responsive policyP0,and forD>Kuse the green-times generated byP0whenD=K.Thus in this modification the green-time proportions generated by responsive P0 are frozen as demand passes through K; and remain fixed for all demands D which exceed K. The paper shows that the above modification when applied to a continuous version of MP also maximises throughput in both networks for all values of the steady demand D. Finally the paper considers a signal controlled network with route choice. It is shown that if routes are chosen by travellers in their own best interests then the continuous version of MP does not always maximise throughput, even if the constant input demand D is within the network capacity, but P0 does always maximise throughput, even when the constant input demand D is beyond the network capacity.

Cooperative traffic signal control using Multi-step return and Off-policy Asynchronous Advantage Actor-Critic Graph algorithm

Intelligent traffic signal control helps to reduce traffic congestion and thus has been studied for a few decades. Multi-intersection cooperative traffic signal control (CTSC), which is more practical than single-intersection traffic signal control, has attracted much attention and research in recent years. Existing works on multi-intersection CTSC make responsive policies based on the sequence of agents’ actions. One issue in multi-intersection CTSC is that every agent’s actions are mapped from its own road information and some useful information, e.g., the distance of adjacent agents, is ignored, which may lead to suboptimal traffic signal control policies. To address this issue, in this paper a decentralized coordination graph algorithm, referred to as Multi-step return and Off-policy Asynchronous Advantage Actor-Critic Graph (MOA3CG) algorithm, is proposed. The MOA3CG algorithm is based on an asynchronous method of multiagent deep reinforcement learning and a coordination graph; the proposed algorithm makes traffic signal control policies based on current traffic states, the history of observations and other information. A new reward function and An Adjusting Matrix of Traffic Signal Phase Control (AMTSPC) are proposed, which are used by the MOA3CG algorithm in the policy-making process; the AMTSPC is to alter selection of actions by considering the distance of adjacent agents. Experimental results on real-world road scenarios show that the proposed algorithm outperforms other four state-of-the-art algorithms in terms of average delay, average traveling time of vehicles, and the throughput of vehicles, thus eventually helps to mitigate traffic congestion.

On Generalized Proportional Allocation Policies for Traffic Signal Control

The fast-increasing demand and relatively slow growth of infrastructure capacity are providing a strong motivation for research in real-time urban traffic controls that make the best use of novel sensing in order to increase efficiency and resilience of the transportation system. In our contribution, we focus on a class of dynamic feedback traffic signal control policies that are based on a generalized proportional allocation rule. The proposed traffic signal controls are decentralized (they make use of local information only), scalable (they are independent of the network size and topology), and universal (they do not rely on any information about external inflows or turning ratios). In spite of their fully distributed nature, we prove that such control policies achieve a global objective, maximum throughput, in that they stabilize the urban traffic network whenever possible under the given capacity constraints.The traffic model we consider consists in a network of interconnected vertical queues with deterministic dynamics driven by physical laws (conservation of mass and preservation of non-negativity of the traffic volumes) as well as scheduling constraints (described as a set of phases, each phase consisting in a subset of lanes that can be be given green light simultaneously). This results in a differential inclusion for which we prove existence and, in the special case of orthogonal phases, uniqueness of continuous solutions via a generalization of the reflection principle. Stability is then proved by interpreting the generalized proportional allocation controllers as minimizers of a certain entropy-like function that is then used as a Lyapunov function for the closed-loop system.

Forward search algorithm based on dynamic programming for real‐time adaptive traffic signal control

The scheduling of traffic signal at intersections is involved in an application of artificial intelligence system. This study presents a new forward search algorithm based on dynamic programming (FSDP) under a decision tree, and explores an efficient solution for real-time adaptive traffic signal control policy. Traffic signal control with cases of fixed phase sequence and variable phase sequence are both considered in the algorithm. Owing to the properties of forward research dynamic programming and the process optimisation of repeated or invalid traffic states the authors proposed, FSDP algorithm reduces the number of states and saves much computation time. Consequently, FSDP is certain to be an on-line algorithm through its application to a complicated traffic control problem. Moreover, the labelled position method is firstly proposed in the author's study to search the optimal policy after reaching the goal state. For practical operations, this new algorithm is extended by adding the rolling horizon approach, and some derived methods are compared with the optimal fixed-time control and adaptive control on the evaluation of traffic delay. Experimental results obtained by the simulations of symmetrical and asymmetrical traffic flow scenarios show that the FSDP method can perform quite well with high efficiency and good qualities in traffic control.

Decentralized signal control for urban road networks

We propose in this paper a decentralized traffic signal control policy for urban road networks. Our policy is an adaptation of a so-called BackPressure scheme which has been widely recognized in data network as an optimal throughput control policy. We have formally proved that our proposed BackPressure scheme, with fixed cycle time and cyclic phases, stabilizes the network for any feasible traffic demands. Simulation has been conducted to compare our BackPressure policy against other existing distributed control policies in various traffic and network scenarios. Numerical results suggest that the proposed policy can surpass other policies both in terms of network throughput and congestion.

A Traffic Signal Control Policy for Emergency Vehicles Preemption Using Timed Petri Nets

All emergency vehicles are permitted to break conventional road rules in order to reach their destinations in the fastest possible time, such as driving through an intersection when the traffic light is red. This work focuses on the use of Timed Petri nets to model the preemption of emergency vehicles system. The advantage of the proposed approach is the clear presentation of traffic lights’ behaviors in terms of conditions and events that cause the preemption of phase being changed. The resulting models allow one to identify and thus avoid urgent scenarios in such systems by conditions and events of the model that control the phase of traffic light alternations. Moreover, this paper also proposes an emergency vehicle preemption policy that provides the emergency vehicles can pass through the intersections with less delay. The liveness and reversibility of the proposed model are verified.

On Maximally Stabilizing Traffic Signal Control with Unknown Turn Ratios

This paper designs distributed dynamic traffic signal control policies for urban traffic networks. Vehicles at the end of an approach to an intersection queue up in separate lanes corresponding to different possible turn maneuvers at the upcoming intersection, according to fixed turn ratios. The departure rate of vehicles from the queue is governed by traffic signal control at the intersections. We consider traffic signal control architectures under which, at every intersection, only a subset of non-conflicting approaches get green light simultaneously. We propose a class of cooperative green light policies, under which traffic signal control at an intersection requires information only about the occupancy levels on the lanes incoming at that intersection. In particular, such a policy does not require information about turn ratios, external arrival rates or departure rates. We show that such minimalist policies are maximally stabilizing for acyclic network topologies under some scenarios, and, when the network is stabilizable, the network admits a globally asymptotically stable equilibrium under these policies. Simulations to illustrate the applicability of our results to cyclic network topologies are also presented.

Reinforcement learning-based multi-agent system for network traffic signal control

A challenging application of artificial intelligence systems involves the scheduling of traffic signals in multi-intersection vehicular networks. This paper introduces a novel use of a multi-agent system and reinforcement learning (RL) framework to obtain an efficient traffic signal control policy. The latter is aimed at minimising the average delay, congestion and likelihood of intersection cross-blocking. A five-intersection traffic network has been studied in which each intersection is governed by an autonomous intelligent agent. Two types of agents, a central agent and an outbound agent, were employed. The outbound agents schedule traffic signals by following the longest-queue-first (LQF) algorithm, which has been proved to guarantee stability and fairness, and collaborate with the central agent by providing it local traffic statistics. The central agent learns a value function driven by its local and neighbours' traffic conditions. The novel methodology proposed here utilises the Q-Learning algorithm with a feedforward neural network for value function approximation. Experimental results clearly demonstrate the advantages of multi-agent RL-based control over LQF governed isolated single-intersection control, thus paving the way for efficient distributed traffic signal control in complex settings.

IMPACTS OF GROUP-BASED SIGNAL CONTROL POLICY ON DRIVER BEHAVIOR AND INTERSECTION SAFETY

In this paper, the authors evaluate the impacts of a group-based traffic signal control policy on driver behavior and intersection safety in Japan. Also known as movement-based, this policy refers to a control patten in which the controller is able to separately allocate time to each signal group instead of allocating based on traffic demand. Using data collected at an intersection before and after such a policy was implemented, the authors draw conclusions regarding the potential safety benefits of implementing such a policy. In addition, the authors discuss conclusion regarding the influence of group-based control policy on the occurrence probability and rate of red light running (RLR) and yellow entry time at intersections.

A New Design Procedure for Traffic Signal Control Policies

A procedure for the design of traffic signal control policies which results in minimal average delay and queue length is presented. The procedure is applicable to preset and sampled data controllers. The optimal policy is computed from measurements of the average demand in each flow direction during some interval (10 minutes to several hours). In many cases, substantial improvements over classical techniques in the areas of intersection performance and safety are obtained. In other cases, the theoretical foundations of the new procedure provide proof that the classical design techniques already generate optimal solutions. Computation time for the procedure is sufficiently small to allow a single automaton to control many (100-1000) intersections in real time.