Traffic Signal Control Algorithm Research Articles

Deep Q-learning Approach based on CNN and XGBoost for Traffic Signal Control

Traffic signal control is a way for reducing traffic jams in urban areas, and to optimize the flow of vehicles by minimizing the total waiting times. Several intelligent methods have been proposed to control the traffic signal. However, these methods use a less efficient road features vector, which can lead to suboptimal controls. The objective of this paper is to propose a deep reinforcement learning approach as the hybrid model that combines the convolutional neural network with eXtreme Gradient Boosting to traffic light optimization. We first introduce the deep convolutional neural network architecture for the best features extraction from all available traffic data and then integrated the extracted features into the eXtreme Gradient Boosting model to improve the prediction accuracy. In our approach; cross-validation grid search was used for the hyper-parameters tuning process during the training of the eXtreme Gradient Boosting model, which will attempt to optimize the traffic signal control. Our system is coupled to a microscopic agent-based simulator (Simulation of Urban MObility). Simulation results show that the proposed approach improves significantly the average waiting time when compared to other well-known traffic signal control algorithms.

Deep Q learning-based traffic signal control algorithms: Model development and evaluation with field data

To contend traffic congestion on urban networks, existing studies have made great efforts to develop traffic-responsive signal timing algorithms in the last decade. More recently, as an alternative to conventional model-based algorithms, machine learning-based methods have been tested on traffic light timing problems and show promising potentials. However, many researchers and practitioners still questioned the feasibility and applicability of adopting machine learning techniques in the ATSC domain. One of the reasons is that these methods assumed flawless detectors and heavily relied on simulators for training and evaluations. To address such a critical concern, this article customizes a Deep Q-learning Learning (DQL) method to optimize traffic signal timings at urban intersections, where the partial observations from identity-based detectors are inputs, and the green splits are outputs. A simulation-free data-driven prediction model is also developed to train the DQL with reduced computational time. Then the machine learning-based methods are evaluated on a real-world case with Automatic Number-Plate Recognition (ANPR) data. Experiments show the proposed data-driven model can predict the traffic state in limited computational time, and the DQL algorithm is 3.9% better than the field experiment performance from the adaptive control system, SCOOT, and 22% better than the time-of-day plan by SYNCHRO. The results indicate the DQL methods can only yield marginal improvement with restrictive input and output settings in congested traffic flow in comparison to the conventional adaptive method.

Real-Time Crash-Risk Optimization at Signalized Intersections

Over the past few decades, numerous adaptive traffic signal control (ATSC) algorithms have been proposed to alleviate traffic congestion and optimize traffic mobility using real-time traffic data, such as data from connected vehicles (CVs). However, most of the existing ATSC algorithms do not consider optimizing traffic safety, likely because of the lack of tools to evaluate safety in real time. In this paper, we propose a novel ATSC algorithm for real-time safety optimization. The algorithm utilizes a traditional Reinforcement Learning approach (i.e., Q-learning) as well as recently developed extreme value theory (EVT) real-time crash prediction models. The algorithm was validated using real-world traffic video data collected from two signalized intersections in British Columbia. The results indicated that, compared with an existing fully actuated signal controller, the developed algorithm can significantly reduce the real-time crash risk by 43% to 45% at the intersection’s approaches even at low CVs market penetration rates.

Adaptive Traffic Management System using CNN (YOLO)

The huge number of vehicles on the roadways is making congestion a significant problem. The line longitudinal vehicle waiting to be processed at the crossroads increases quickly, and the traditionally used traffic signals are not able to program it properly. Manual traffic monitoring may be an onerous job since a number of cameras are deployed over the network in traffic management centers. The proactive decision-making of human operators, which would decrease the effect of events and recurring road congestion, might contribute to the easing of the strain of automation.The traffic control frameworks in India are now needed as it is an open-loop control framework, without any input or detection mechanism. Inductive loops and sensors employed in existing technology used to detect the number of passing vehicles. The way traffic lights are adapted is highly inefficient and costly in this existing technology. The aim was to build a traffic control framework by introducing a system for detection ,which gives an input to the existing system (closed loop control system) in order to adapt to the changing traffic density patterns and to provide the controller with a crucial indication for ongoing activities. By this technique, the improvement of the signals on street is extended and thus saves time by preventing traffic congestion. This study proposes an algorithm for real-time traffic signal control, depending on the traffic flow. In reality, the features of competitive traffic flow at the signposted road crossing are used by computer vision and by machine learning. This is done by the latest, real-time object identification, based on convolutional Neural Networks network called You Look Once (YOLO). Traffic signal phases are then improved by data acquired in order to allow more vehicles to pass safely over minimal wait times, particularly the line long and the time of waiting per vehicle.This adjustable traffic signal timer is used to calculate traffic density utilizing YOLO object identification using live pictures of cameras in intervals and adjusts the signal timers appropriately, therefore decreasing the road traffic congestion, ensuring speedier transit for persons, and reducing fuel consumption. The traffic conditions will improve enormously at a relatively modest cost. Inductive loops are a viable but costly approach. This method thereby cuts expenses and outcomes quickly.

IoT-based traffic prediction and traffic signal control system for smart city

Because of the population increasing so high, and traffic density remaining the same, traffic prediction has become a great challenge today. Creating a higher degree of communication in automobiles results in the time wastage, fuel wastage, environmental damage, and even death caused by citizens being trapped in the middle of traffic. Only a few researchers work in traffic congestion prediction and control systems, but it may provide less accuracy. So, this paper proposed an efficient IoT-based traffic prediction using OWENN algorithm and traffic signal control system using Intel 80,286 microprocessor for a smart city. The proposed system consists of ‘5’ phases, namely IoT data collection, feature extraction, classification, optimized traffic IoT values, and traffic signal control system. Initially, the IoT traffic data are collected from the dataset. After that, traffic, weather, and direction information are extracted, and these extracted features are given as input to the OWENN classifier, which classifies which place has more traffic. Suppose one direction of the place has more traffic, it optimizes the IoT values by using IBSO, and finally, the traffic is controlled by using Intel 80,286 microprocessor. An efficient OWENN algorithm for traffic prediction and traffic signal control using a Intel 80,286 microprocessor for a smart city. After extracting the features, the classification is performed in this step. Hereabout, the classification is done by using the optimized weight Elman neural network (OWENN) algorithm that classifies which places have more traffic. OWENN attains 98.23% accuracy than existing model also its achieved 96.69% F-score than existing model. The experimental results show that the proposed system outperforms state-of-the-art methods.

IHG-MA: Inductive heterogeneous graph multi-agent reinforcement learning for multi-intersection traffic signal control

Multi-agent deep reinforcement learning (MDRL) has been widely applied in multi-intersection traffic signal control. The MDRL algorithms produce the decentralized cooperative traffic-signal policies via specialized multi-agent settings in certain traffic networks. However, the state-of-the-art MDRL algorithms seem to have some drawbacks. (1) It is desirable that the traffic-signal policies can be smoothly transferred to diverse traffic networks, however, the adopted specialized multi-agent settings hinder the traffic-signal policies to transfer and generalize to new traffic networks. (2) Existing MDRL algorithms which are based on deep neural networks cannot flexibly tackle a time-varying number of vehicles traversing the traffic networks. (3) Existing MDRL algorithms which are based on homogeneous graph neural networks fail to capture the heterogeneous features of objects in traffic networks. Motivated by the above observations, in this paper, we propose an algorithm, referred to as Inductive Heterogeneous Graph Multi-agent Actor–critic (IHG-MA) algorithm, for multi-intersection traffic signal control. The proposed IHG-MA algorithm has two features: (1) It conducts representation learning using a proposed inductive heterogeneous graph neural network (IHG), which is an inductive algorithm. The proposed IHG algorithm can generate embeddings for previously unseen nodes (e.g., new entry vehicles) and new graphs (e.g., new traffic networks). But unlike the algorithms based on the homogeneous graph neural network, IHG algorithm not only encodes heterogeneous features of each node, but also encodes heterogeneous structural (graph) information. (2) It also conducts policy learning using a proposed multi-agent actor–critic(MA), which is a decentralized cooperative framework. The proposed MA framework employs the final embeddings to compute the Q-value and policy, and then optimizes the whole algorithm via the Q-value and policy loss. Experimental results on different traffic datasets illustrate that IHG-MA algorithm outperforms the state-of-the-art algorithms in terms of multiple traffic metrics, which seems to be a new promising algorithm for multi-intersection traffic signal control.

Security Vulnerabilities and Protection Algorithms for Backpressure-Based Traffic Signal Control at an Isolated Intersection

There is an increasing trend in the use of wireless communication along with new traffic signal control (TSC) algorithms to leverage and accommodate connected and autonomous vehicles. However, this development has increased the potential for cyber-attacks on TSC that can undermine the benefits of these new algorithms. An advanced persistent adversary can learn the behavior of TSC algorithms and launch attacks to preferentially get green time and/or to create traffic congestion in one intersection which can spread to the entire network. In this paper, we consider backpressure-based (BP-based) TSC algorithms and compare their performance under two misinformation attacks - 1) time spoofing attack in which vehicles alter their arrival times at the intersection and 2) ghost vehicle attack in which vehicles disconnect the wireless communication and thereby hide from the TSC. We show that these misinformation can influence the signal phases determined by BP-based TSC algorithms. We consider an adversary that determines a set of arriving vehicles to be attack vehicles from many candidate sets (attack strategies) in order to maximize the number of disrupted signal phases. We show that by formulating the problem as a 0/1 Knapsack problem, the adversary can explore the space of attack strategies and determine the optimal strategy that maximally compromises the performance in terms of average delay and fairness. We propose two protection algorithms, namely, auction-based (APA) and hybrid-based (HPA) algorithms and show that they are able to mitigate the impacts of the misinformation attacks.

Adversarial Attacks and Defense in Deep Reinforcement Learning (DRL)-Based Traffic Signal Controllers

Security attacks on intelligent transportation systems (ITS) may result in life-threatening situations. Combining deep neural networks with reinforcement learning (RL) models called DRL shows promising results when applied to urban Traffic Signal Control (TSC) for adaptive adjustment of traffic light schedules. In this paper, first, we explore the security vulnerabilities of DRL-based TSCs in the presence of adversarial attacks. We investigate the impact of the two distinct threat models with two state-of-the-art adversarial attacks using white-box and black-box settings. The attacks are simulated on different DRL-based TSC algorithms in a single intersection and multiple intersections. The results show that the performance of the DRL learning agent decreases in both adversarial attack models with white-box and black-box settings resulting in higher levels of traffic congestion. After analysing the adversarial attack models, we explored several sequential anomaly detection models. While sequential anomaly detection models minimizes the detection delays, it also achieves lower false alarm rates due to cumulative anomaly inspection. We also proposed an ensemble model that works with all the attack models without any model assumption. The results of anomaly detectors indicates that low-cost ensemble model achieves the best anomaly detection performance in all attack models and DRL settings.

A semi-decentralized feudal multi-agent learned-goal algorithm for multi-intersection traffic signal control

Traffic signal control helps to reduce traffic congestion and thus has been studied for a few decades. Algorithms for multi-intersection traffic signal control that adopt hierarchical deep reinforcement learning have been shown to achieve state-of-the-art performance. For these methods, a two-level hierarchical structure is generally used, in which the low-level policies employ latent goals induced by the high-level policies. Existing research either designs the latent goals manually or acquires the latent goals from the environment. However, these latent goals are not optimal, which leads to the non-optimal low-level policies. To improve this, we propose a learned-goal soft actor–critic (LSAC) algorithm, by which the optimal latent goals are automatically learned and then are used in low-level policies. We then propose a semi-decentralized feudal multi-agent (SFM) framework, which can alleviate the problem that existing multi-agent framework faces, i.e., the fast growing state space with the increase of the number of agents. Combining the above two proposed methods, an SFM-LSAC algorithm is proposed in this paper for multi-intersection traffic signal control. Other techniques are also incorporated into the SFM-LSAC algorithm such as attention mechanism. Experimental results in three real-world traffic scenarios illustrate that the SFM-LSAC algorithm outperforms other four state-of-the-art multi-intersection traffic signal control algorithms, i.e., reduces the average intersection delay, the average queue length, the average travel time, meanwhile improves the average flow and average travel speed, thus could be a new promising algorithm for multi-intersection traffic signal control.

BACKPRESSURE BASED TRAFFIC SIGNAL CONTROL CONSIDERING CAPACITY OF DOWNSTREAM LINKS

Congestion is a kind of expression of instability of traffic network. Traffic signal control keeping traffic network stable can reduce the congestion of urban traffic. In order to improve the efficiency of urban traffic network, this study proposes a decentralized traffic signal control strategy based on backpressure algorithm used in Wi-Fi mesh networks for packets routing. Backpressure based traffic signal control algorithm can stabilize urban traffic network and achieve maximum throughput. Based on original backpressure algorithm, the variant parameter and penalty function are considered to balance the queue differential and capacity of downstream links in urban traffic network. For each traffic phase of intersections, phase weight is computed using queue differential and capacity of downstream links, which fixed the deficiency of infinite queue capacity in original backpressure algorithm. It is proved that the extended backpressure traffic signal control algorithm can maintain stability of urban traffic network, and also can prevent queue spillback, so as to improve performance of whole traffic network. Simulations are carried out in Vissim using Vissim COM programming interface and Visual Studio development tools. Evaluation results illuminate that it can get better performance than the backpressure algorithm just based on queue length differential in average queue length and delay of traffic network.

Adaptive traffic signal control algorithms based on probe vehicle data

In this paper, two new adaptive traffic signal control algorithms are proposed based on data from probe vehicles to realize the coordinated signal control of arterial roads. One is an iterative signal control algorithm, and the other is an optimized signal control algorithm. The proportion of vehicles in the nonstop group is selected as the indicator of the traffic state. The value for this indicator can be accurately estimated by data from probe vehicles. Our goal is to ease traffic congestion and enhance the capacities of traffic networks. Compared with the Webster fixed-time signal control algorithm, these two new adaptive signal control algorithms are evaluated on a microscopic simulation platform. Simulation results show that the average travel time is reduced by 32% under the iterative signal control algorithm and by 23% under the optimized signal control algorithm, and the average delay times are reduced by 36% and 35%, respectively. In the meantime, the average number of stops under the iterative signal control algorithm is reduced by 43%, and under the optimized signal control algorithm, by 67%. They indicate that the two new adaptive signal control algorithms are effective for easing traffic congestion and achieve the adaptive signal control objectives using real-time traffic information.

A simple crowdsourced delay-based traffic signal control.

Current transportation management systems rely on physical sensors that use traffic volume and queue-lengths. These physical sensors incur significant capital and maintenance costs. The ubiquity of mobile devices has made possible access to accurate and cheap traffic delay data. However, current traffic signal control algorithms do not accommodate the use of such data. In this paper, we propose a novel parsimonious model to utilize real-time crowdsourced delay data for traffic signal management. We demonstrate the versatility and effectiveness of the data and the proposed model on seven different intersections across three cities and two countries. This signal system provides an opportunity to leapfrog from physical sensors to low-cost, reliable crowdsourced data.

Augmenting Traffic Signal Control Systems for Urban Road Networks With Connected Vehicles

The increase in traffic volumes in urban areas makes network delay and capacity optimisation challenging. However, the introduction of connected vehicles in intelligent transport systems presents unique opportunities for improving traffic flow and reducing delays in urban areas. This paper proposes a novel traffic signal control algorithm called Multi-mode Adaptive Traffic Signals (MATS) which combines position information from connected vehicles with data obtained from existing inductive loops and signal timing plans in the network to perform decentralised traffic signal control at urban intersections. The MATS algorithm is capable of adapting to scenarios with low numbers of connected vehicles, an area where existing traffic signal control strategies for connected environments are limited. Additionally, a framework for testing connected traffic signal controllers based on a large urban road network in the city of Birmingham (UK) is presented. The MATS algorithm is compared with MOVA on a single intersection, and a calibrated TRANSYT plan on the proposed testing framework. The results show that the MATS algorithm offers reductions in mean delay up to 28% over MOVA, and reductions in mean delay and mean numbers of stops of up to 96% and 33% respectively over TRANSYT, for networks with 0-100% connected vehicle presence. The MATS algorithm is also shown to be robust under non-ideal communication channel conditions, and when heavy traffic demand prevails on the road network.

Double Deep Q-Network with a Dual-Agent for Traffic Signal Control

Adaptive traffic signal control (ATSC) based on deep reinforcement learning (DRL) has shown promising prospects to reduce traffic congestion. Most existing methods keeping traffic signal phases fixed adopt two agent actions to match a four-phase suffering unstable performance and undesirable operation in a four-phase signalized intersection. In this paper, a Double Deep Q-Network (DDQN) with a dual-agent algorithm is proposed to obtain a stable traffic signal control policy. Specifically, two agents are denoted by two different states and shift the control of green lights to make the phase sequence fixed and control process stable. State representations and reward functions are presented by improving the observability and reducing the leaning difficulty of two agents. To enhance the feasibility and reliability of two agents in the traffic control of the four-phase signalized intersection, a network structure incorporating DDQN is proposed to map states to rewards. Experiments under Simulation of Urban Mobility (SUMO) are carried out, and results show that the proposed traffic signal control algorithm is effective in improving traffic capacity.

Research and application of intersection traffic signal control algorithm based on vehicle location

This paper focuses on the optimising algorithms of signal timing of urban intersection combined with vehicle location using the platform of the new type of wireless electronic license plate system. With the traffic flow information obtained from the platform, a dynamic optimisation model with optimal control parameters of green light time of all four phases for single intersection is constructed. Then genetic algorithm is applied to obtain the satisfying signal timing scheme. In simulation examples, the proposed dynamic distribution method and the traditional time control method are compared. The results show that compared with the traditional time control method, in traffic rush hour, the genetic optimisation algorithm can reduce the waiting time by 23% with the same road vehicle capacity. At non-peak time, the proposed method can reduce the waiting time by 10%. Therefore, the introduced method can effectively reduce the total queue delay, and improve the traffic capacity of the intersection.

PlanLight: Learning to Optimize Traffic Signal Control With Planning and Iterative Policy Improvement

Intelligent traffic signal control (TSC) is essential for transportation efficiency in modern road networks. There is an emerging trend of using deep reinforcement learning techniques to train TSC models in simulators for reducing trial-and-error in real-world scenarios, and recent studies have shown promising results. The target of TSC is to minimize the average travel time of a given area. However, it is impractical to directly optimize the target by setting the average travel time as the reward function due to its nature of feedback latency and difficulty on credit assignment. Existing methods often define the reward function in a heuristic way, which may cause a biased optimization on the real target as they only optimize the accumulative reward. In this work, we propose PlanLight, a novel planning-based TSC algorithm that learns from the demonstration of rollout algorithm, which obtains suboptimal control on the given target, through behavior cloning. We show the effectiveness and efficiency of the rollout algorithm in the multi-intersection control scenario. Moreover, we achieve further policy optimization by improving the base policy in the rollout procedure iteratively. Through comprehensive experiments, we demonstrate that PlanLight outperforms both conventional transportation approaches and existing learning-based methods in various sizes of traffic datasets. Furthermore, we empirically show the potential of PlanLight to be a general algorithm to obtain improvement on future state-of-the-art TSC methods.

A heuristic method to optimize generic signal phasing and timing plans at signalized intersections using Connected Vehicle technology

This paper develops a flexible, real-time traffic signal control algorithm to optimize both phase durations and phase sequences at four-approach intersections with conflicting left-turns, based on information obtained from connected vehicles. Vehicle-to-infrastructure communications are assumed to provide the location of all connected vehicles near the signalized intersection at regular time intervals and this information is used to identify the presence of non-connected vehicles that are stopped at the intersection. All detected and identified vehicles are used to identify naturally occurring platoons in the traffic stream. The signal control algorithm then selects the optimal sequence that these platoons should discharge through the intersection to minimize average delay of all identified vehicles. Since all possible departure sequences for platoons of vehicles are considered, the problem is computationally difficult. Hence, several heuristic methods are proposed to determine optimal platoon departure sequences. These heuristics include an intelligent tree search and multiple types of genetic algorithms, including a newly developed genetic algorithm that perserves phase order sequence that is vital to this problem. Comparisons between these heuristics and the global optimal solution suggest that the heuristics are able to provide similar operational performance with significant reductions in total computation time required, such that the algorithm can be applied in real-time. In general, the intelligent tree search appears to outperform the genetic algorithm approaches in terms of operational performance but has computational requirements that increase exponentially with the number of platoons identified at the intersection. Meanwhile, the genetic algorithm methods tend to be more scalable but slightly less efficient. Overall, the results are promising for the application of the proposed flexible signal control algorithm at real intersections.

Cooperative Traffic Signal Control with Traffic Flow Prediction in Multi-Intersection.

As traffic congestion in cities becomes serious, intelligent traffic signal control has been actively studied. Deep Q-Network (DQN), a representative deep reinforcement learning algorithm, is applied to various domains from fully-observable game environment to traffic signal control. Due to the effective performance of DQN, deep reinforcement learning has improved speeds and various DQN extensions have been introduced. However, most traffic signal control researches were performed at a single intersection, and because of the use of virtual simulators, there are limitations that do not take into account variables that affect actual traffic conditions. In this paper, we propose a cooperative traffic signal control with traffic flow prediction (TFP-CTSC) for a multi-intersection. A traffic flow prediction model predicts future traffic state and considers the variables that affect actual traffic conditions. In addition, for cooperative traffic signal control in multi-intersection, each intersection is modeled as an agent, and each agent is trained to take best action by receiving traffic states from the road environment. To deal with multi-intersection efficiently, agents share their traffic information with other adjacent intersections. In the experiment, TFP-CTSC is compared with existing traffic signal control algorithms in a 4 × 4 intersection environment. We verify our traffic flow prediction and cooperative method.

Max-pressure traffic controller based on travel times: An experimental analysis

The traffic control of an arbitrary network of signalized intersections is considered. This work presents a new version of the recently proposed max-pressure controller, also known as back-pressure. The most remarkable features of the max-pressure algorithm for traffic signal control are: scalability, stability, and distribution. The modified version presented in this paper improves the practical applicability of the max-pressure controller by considering as input travel times instead of queue lengths. The two main practical advantages of this new version are: (i) travel times are easier to estimate than queue lengths, and (ii) max-pressure controller based on travel times has an inherent capacity-aware property, i.e., it takes into account the finite capacity of each link. Travel time tends to diverge when the queue length is close to its capacity. It should be noted that previous max-pressure algorithms rely exclusively on queue length measurements, which may be difficult to accomplish in practice. Moreover, these previous algorithms generally assume queues with unbounded capacity. This may be problematic because a model with unbounded capacity links is not able to reproduce spillbacks, which are one of the most critical phenomena that a traffic signal controller should avoid. After presenting the new version of the max-pressure controller, it is analyzed and compared with existing control policies in a microscopic traffic simulator. Moreover, results of a real implementation of the developed algorithm to a signalized intersection, located at an urban arterial in Jerusalem, are shown and analyzed. To the best of the authors’ knowledge, this experiment is the first real implementation of a max-pressure controller at a signalized intersection.

An equitable traffic signal control scheme at isolated signalized intersections using Connected Vehicle technology

This paper extends a real-time, Connected Vehicle-based traffic signal control algorithm at isolated intersections to balance between two competing intersection objectives: efficiency and equity. In this approach, a central controller is used to collect real-time locations of connected vehicles at regular intervals, which can then be used to also identify the existence of some of the non-connected vehicles. The control algorithm then aims to optimize the discharge sequence of naturally occurring platoons of vehicles based on their proximity. Specifically, the strategy selects the platoon discharge sequence—and associated signal phase and timing plan—that minimizes average vehicle delay (measure of efficiency) while limiting the maximum delay any individual vehicle may experience (measure of equity). The latter objective is only possible with detailed vehicle-level information available from connected vehicles. The results show that without the threshold on maximum individual vehicle delay, average delay is often minimized at the expense of very large delays imposed onto some vehicles. By implementing a threshold, both the maximum vehicle delay and the distribution of individual vehicle delays—as measured by the standard deviation and a common measure of population inequity, the Gini coefficient—can be improved, often with only negligible impacts to intersection efficiency. However, the tradeoff between equity and efficiency becomes more significant as the maximum vehicle delay threshold decreases. Sensitivity tests show that this control algorithm works well for different total traffic demands and different demand patterns. The proposed algorithm is also effective under imperfect connected vehicle penetration rates when the connected vehicles make up more than 40% of the traffic stream. The results suggest that the proposed strategy can help significantly reduce long delays and inequitable treatment of vehicles at an intersection when vehicle-level information is available to a signal controller.