Traffic Signal Control Method Research Articles

F rugal L ight : Symmetry-Aware Cyclic Heterogeneous Intersection Control using Deep Reinforcement Learning with Model Compression, Distillation and Domain Knowledge

Developing countries need to better manage fast increasing traffic flows, owing to rapid urbanization. Else, increasing traffic congestion would increase fatalities due to reckless driving, as well as keep vehicular emissions and air pollution critically high in cities like New Delhi. State-of-the-art traffic signal control methods in developed countries, however, use expensive sensing, computation, and communication resources. How far can control algorithms go, under resource constraints, is explored through the design and evaluation of FrugalLight (FL) in this article. We also captured and processed a real traffic dataset at a busy intersection in New Delhi, India, using efficient techniques on low cost embedded devices. This dataset ( https://delhi-trafficdensity-dataset.github.io ) contains traffic density information at fine time granularity of one measurement every second, from all approaches of the intersection for 40 days. FrugalLight ( https://github.com/sachin-iitd/FrugalLight ) is evaluated on the collected traffic dataset from New Delhi and another open source traffic dataset from New York. FrugalLight matches the performance of state-of-the-art Convolutional Neural Network (CNN) based sensing and Deep Reinforcement Learning (DRL) based control algorithms, while utilizing resources less by an order of magnitude. We further explore improvements using a careful combination of knowledge distillation and domain knowledge based DRL model compression, with employing Model-Agnostic Meta-Learning to quickly adapt to traffic at new intersections. The collected real dataset and FrugalLight therefore opens up opportunities for resource efficient RL based intersection control design for the ML research community, where the controller should have limited carbon footprint. Such intelligent, green, intersection controllers can help reduce traffic congestion and associated vehicular emissions, even if compute and communication infrastructure is limited in low resource regions. This is a critical step toward achieving two of the United Nations Sustainable Development Goals (SDG), namely sustainable cities and communities and climate action.

Deep Reinforcement Learning for Ecological and Distributed Urban Traffic Signal Control with Multi-Agent Equilibrium Decision Making

Multi-Agent Reinforcement Learning (MARL) has shown strong advantages in urban multi-intersection traffic signal control, but it also suffers from the problems of non-smooth environment and inter-agent coordination. However, most of the existing research on MARL traffic signal control has focused on designing efficient communication to solve the environment non-smoothness problem, while neglecting the coordination between agents. In order to coordinate among agents, this paper combines MARL and the regional mixed-strategy Nash equilibrium to construct a Deep Convolutional Nash Policy Gradient Traffic Signal Control (DCNPG-TSC) model, which enables agents to perceive the traffic environment in a wider range and achieves effective agent communication and collaboration. Additionally, a Multi-Agent Distributional Nash Policy Gradient (MADNPG) algorithm is proposed in this model, which is the first time the mixed-strategy Nash equilibrium is used for the improvement in the Multi-Agent Deep Deterministic Policy Gradient algorithm traffic signal control strategy to provide the optimal signal phase for each intersection. In addition, the eco-mobility concept is integrated into MARL traffic signal control to reduce pollutant emissions at intersections. Finally, simulation results in synthetic and real-world traffic road networks show that DCNPG-TSC outperforms other state-of-the-art MARL traffic signal control methods in almost all performance metrics, because it can aggregate the information of neighboring agents and optimize the agent’s decisions through gaming to find an optimal joint equilibrium strategy for the traffic road network.

Traffic Light Control System

The traffic light control system is a key component of urban traffic management, and its performance directly impacts the smooth flow of traffic and traffic safety. Based on some issues existing in the current traffic signal control systems, this paper proposes a novel traffic light control system based on intelligent algorithms. This system combines artificial intelligence technology with traditional traffic signal control methods to achieve intelligent control of traffic signals, thereby enhancing traffic efficiency and road safety. The paper provides a detailed explanation of the system's design principles, algorithm implementation, and experimental results. Comparative experiments demonstrate that the system has significant advantages in reducing traffic congestion and improving vehicle passage efficiency. Finally, the paper discusses possible future improvement directions and application scenarios, envisioning the development prospects of traffic light control systems in the future.

CVDMARL: A Communication-Enhanced Value Decomposition Multi-Agent Reinforcement Learning Traffic Signal Control Method

Effective traffic signal control (TSC) plays an important role in reducing vehicle emissions and improving the sustainability of the transportation system. Recently, the feasibility of using multi-agent reinforcement learning technology for TSC has been widely verified. However, the process of mapping road network states onto actions has encountered many challenges, due to the limited communication between agents and the partial observability of the traffic environment. To address this problem, this paper proposes a communication-enhancement value decomposition, multi-agent reinforcement learning TSC method (CVDMARL). The model combines two communication methods: implicit and explicit communication, decouples the complex relationships among the multi-signal agents through the centralized-training and decentralized-execution paradigm, and uses a modified deep network to realize the mining and selective transmission of traffic flow features. We compare and analyze CVDMARL with six different baseline methods based on real datasets. The results show that compared to the optimal method MN_Light, among the baseline methods, CVDMARL’s queue length during peak hours was reduced by 9.12%, the waiting time was reduced by 7.67%, and the convergence algebra was reduced by 7.97%. While enriching the information content, it also reduces communication overhead and has better control effects, providing a new idea for solving the collaborative control problem of multi-signalized intersections.

Study of medium and long-term free flow capacity and queue discharge rates on roads.

With the rise in vehicle ownership, traffic congestion has emerged as a major barrier to urban progress, making the study and optimization of urban road capacity exceedingly crucial. The research on the medium and long-term free-flowing capacity and queue emission rate of roads takes an in-depth exploration of this issue from a cutting-edge perspective, aiming to find solutions adaptable to the progression of the times. The purpose of this study is to understand and predict the road capacity and queue emission rate more accurately, thus improving the urban traffic condition. Existing literature primarily focuses on short-term forecasts of road capacity, leaving a notable void in the research of medium and long-term road capacity and queue emission rate. This gap often results in a lack of sufficient foresight when urban traffic planning faces practical issues. To fill this void, this study undertook an in-depth examination of the road capacity and queue emission rate over the medium and long term (10 years) based on big data analysis and artificial intelligence theories. This paper employs a Radial Basis Function (RBF) neural network, combined with twelve other parameters that could potentially impact road capacity, such as traffic volume, road width, number of lanes, traffic signal control methods, etc., to analyze the relationship between each parameter and free-flow traffic and queue emission rate. These analyses are grounded in extensive road data, encompassing not only the city's main roads but also secondary roads and community roads. The study results show a continuous downward trend in the free-flowing capacity of roads and a slight upward trend in the queue emission rate over the past decade. Further analysis reveals the extent of impact each factor has on the free-flow traffic and queue emission rate, providing a scientific basis for future urban traffic planning.

A survey on deep reinforcement learning approaches for traffic signal control

In the domain of complex urban traffic networks, real-time Traffic Signal Control (TSC) serves as a pivotal strategy for mitigating congestion. Coordinating signal control across multiple intersections involves considerable complexity. Deep Reinforcement Learning (DRL) has emerged as a robust solution. In recent years, there has been rapid advancement in TSC methods, with numerous researchers employing various novel DRL methodologies. Yet, existing surveys lack timeliness and universality in capturing the latest research. There is a notable gap in current research surveys with respect to the latest developments in TSC. Therefore, the focus of this paper lies in analyzing the most recent papers from the past five years, with the aim to provide a comprehensive and multi-dimensional review of the evolution of DRL in TSC. The survey categorizes current research based on model setups, utilized algorithms, and application scenarios. Finally, this paper highlights potential directions for future TSC research.

KeyLight: Intelligent Traffic Signal Control Method Based on Improved Graph Neural Network

Graph neural network combined with reinforcement learning is one of the most effective traffic signal control methods. However, existing methods fail to pay enough attention to the key information, such as the traffic information of the downstream section in extracting state and the intersection’s own state in aggregating information from neighbor intersections. As a result, adverse reactions like unstable learning and limited performance occur frequently when agents and models focus on interfering information and useless information too much. In this article, we propose KeyLight, an intelligent traffic signal control method based on reinforcement learning by facilitating the attention of the learning algorithm and model to the key information that is usually ignored. In KeyLight, we design a new state representation NOV-LADLE and introduce residual connection in the graph neural network to highlight the importance of the intersection’s state. Experiments show that, in the case of comparable throughput, the proposed KeyLight has been greatly improved and enhanced in performance. Especially, the average travel time can be increased by up to 23.44% on the real-world dataset.

Field‐tested signal controller to mitigate spillover using trajectory data

AbstractTrajectory data from connected vehicles (CVs) provide a continuous and reliable means of obtaining information that can be leveraged to optimize traffic signals. This paper proposes a real‐time traffic signal control method using CV trajectory data as the sole input. The primary goal of the proposed signal control method is to prevent queue spillover, which may significantly decrease the traffic efficiency on urban networks and induce high delays to the travelers. The proposed method formulates the signal control problem via a linear quadratic optimization model, considering the constraints related to the duration and variability of green lights in practical traffic signal control systems. Compared to conventional max‐pressure‐based methods, the optimization model offers enhanced efficiency in handling these constraints, making it highly suitable for real‐life implementation. The proposed method has undergone testing in both simulated environments and real‐world applications. In the simulation experiments, the proposed method has been demonstrated to effectively reduce spillover risks and outperform a conventional max‐pressure‐based approach even when the CV penetration rate is as low as 5%. In the real‐world experiment, the proposed method had been tested in a traffic network around the Beijing Capital International Airport for several months. The severity of spillovers, which was represented by two performance indicators, had been significantly reduced after implementing the proposed method.

Privacy-preserving adaptive traffic signal control in a connected vehicle environment

Although Connected Vehicles (CVs) have demonstrated tremendous potential to enhance traffic operations, they can impose privacy risks on individual travelers, e.g., leaking sensitive information about their frequently visited places, routing behavior, etc. Despite the large body of literature that devises various algorithms to exploit CV information, research on privacy-preserving traffic control is still in its infancy. In this paper, we aim to fill this research gap and propose a privacy-preserving adaptive traffic signal control method using partially connected vehicle data. Specifically, we proposed a privacy-preserving mechanism to protect CV data against three types of attacks: CV collusion attacks, database attacks, and inference attacks. The mechanism leverages secure Multi-Party Computation and differential privacy to aggregate individual-level CV data to calculate key traffic parameters without compromising the privacy of CV users. For seamless integration with the privacy-preserving mechanism, we develop a traffic signal optimization model and an arrival rate estimator relying only on aggregated CV data, being applied to both undersaturated and oversaturated traffic conditions. The optimization model is further extended to a stochastic programming problem to explicitly handle the noises added by the privacy-preserving mechanism. Evaluation results show that the linear optimization model preserves privacy with a marginal impact on control performance, and the stochastic programming model can significantly reduce residual queues compared to the linear programming model, with almost no increase in vehicle delay. Overall, our methods demonstrate the feasibility of incorporating privacy-preserving mechanisms in CV-based traffic modeling and control, which guarantees both utility and privacy.

Improved Deep Reinforcement Learning for Intelligent Traffic Signal Control Using ECA_LSTM Network

Reinforcement learning is one of the most widely used methods for traffic signal control, but the method experiences issues with state information explosion, inadequate adaptability to special scenarios, and low security. Therefore, this paper proposes a traffic signal control method based on the efficient channel attention mechanism (ECA-NET), long short-term memory (LSTM), and double Dueling deep Q-network (D3QN), which is EL_D3QN. Firstly, the ECA-NET and LSTM module are included in order to lessen the state space’s design complexity, improve the model’s robustness, and adapt to various emergent scenarios. As a result, the cumulative reward is improved by 27.9%, and the average queue length, average waiting time, and CO2 emissions are decreased by 15.8%, 22.6%, and 4.1%, respectively. Next, the dynamic phase interval tgap is employed to enable the model to handle more traffic conditions. Its cumulative reward is increased by 34.2%, and the average queue length, average waiting time, and CO2 emissions are reduced by 19.8%, 30.1%, and 5.6%. Finally, experiments are carried out using various vehicle circumstances and unique scenarios. In a complex environment, EL_D3QN reduces the average queue length, average waiting time, and CO2 emissions by at least 13.2%, 20.2%, and 3.2% compared to the four existing methods. EL_D3QN also exhibits good generalization and control performance when exposed to the traffic scenarios of unequal stability and equal stability. Furthermore, even when dealing with unique events like a traffic surge, EL_D3QN maintains significant robustness.

Distributed Stochastic MPC Traffic Signal Control for Urban Networks

In this paper, we design a stochastic model predictive control (MPC)-based traffic signal control method for urban networks when the uncertainties of the traffic model parameters (including the exogenous traffic flows and the turning ratios of downstream traffic flows) are taken into account. Considering that the traffic model parameters are random variables with known expectations and variances, the traffic signal control and coordination problem is formulated as a quadratic program with linear and second-order cone constraints. In order to reduce computational complexity, we suggest a way to decompose the optimization problem corresponding to the whole network into multiple subproblems. By applying an Alternating Direction Method of Multipliers (ADMM) scheme, the optimal stochastic traffic signal splits are found in a distributed manner. The effectiveness of the designed control method is validated via some simulations using VISSIM and MATLAB.

FairLight: Fairness-Aware Autonomous Traffic Signal Control With Hierarchical Action Space

Although Reinforcement Learning (RL) approaches are promising in autonomous Traffic Signal Control (TSC), they often suffer from the unfairness problem that causes extremely long waiting time at intersections for partial vehicles. This is mainly because traditional RL methods focus on optimizing the overall traffic performance, while the fairness of individual vehicles is neglected. To address this problem, we propose a novel RL-based method named FairLight for the fair and efficient control of traffic with variable phase duration. Inspired by the concept of User Satisfaction Index (USI) proposed in the transportation field, we introduce a fairness index in the design of key RL elements, which specially considers the travel quality (e.g., fairness). Based on our proposed hierarchical action space method, FairLight can accurately allocate the duration of traffic lights for selected phases. Experimental results obtained from various well-known traffic benchmarks show that, compared with state-of-the-art RL-based TSC methods, FairLight can not only achieve better fairness performance, but also improve the control quality from the perspectives of the average travel time of vehicles and RL convergence speed.

Cooperative control of multiple intersections combining agent and chaotic particle swarm optimization

The regional intersection can be regarded as an agent model, that should be controlled and optimized. The traditional particle swarm optimization algorithm has been enhanced through optimization techniques and the inclusion of chaos algorithms and agent interactive and cooperative control technology, resulting in the development of a chaotic particle swarm optimization algorithm. Multi-agent is applied to the collaborative control of intersections to realize the collaborative decision-making of multi-intersections. The state and action of intersection signal control agent are defined to reduce the dimension of state-action space and speed up the convergence rate. The proposed method takes vehicle delay, vehicle queue length and regional access ratio as optimization objectives, implements regional traffic signal control strategy and improves overall traffic efficiency. By constructing a simulation environment based on Paramics software, the effectiveness of the regional traffic signal control method based on chaotic particle swarm optimization algorithm is verified, and the convergence efficiency and signal timing optimization have interactive effects.

Traffic signal control using reinforcement learning based on the teacher-student framework

Reinforcement Learning (RL) is an effective method for adaptive traffic signals control. As one type of RL, the teacher-student framework has been found helpful in improving the model performance for different application fields (such as robot control, game, hybrid intelligence), but it is rarely applied for traffic control due to that the hyper-parameters and the number of state-action pairs experienced are difficult to determine. In this work, the teacher-student framework is used for traffic signal control, where only a single reward function is designed to guide the student agent and by using this method the number of hyper-parameters and the model complexity are reduced. Specifically, the teacher agent uses an importance function to evaluate and guide the student, where the importance function combines with environment reward to form a synthetic reward for the student agent. Experimental results under different traffic environments show that the proposed method achieves the expected performance enhancement and is better than most of the state-of-the-art RL-based traffic signal control methods.

Optimal Traffic Signal Control Using Priority Metric Based on Real-Time Measured Traffic Information

Optimizing traffic control systems at traffic intersections can reduce network-wide fuel consumption as well as improve traffic flow. While traffic signals have conventionally been controlled based on predetermined schedules, various adaptive control systems have been developed recently using advanced sensors such as cameras, radars, and LiDARs. By utilizing rich traffic information enabled by the advanced sensors, more efficient or optimal traffic signal control is possible in response to varying traffic conditions. This paper proposes an optimal traffic signal control method to minimize network-wide fuel consumption utilizing real-time traffic information provided by advanced sensors. This new method employs a priority metric calculated by a weighted sum of various factors, including the total number of vehicles, total vehicle speed, vehicle waiting time, and road preference. Genetic Algorithm (GA) is used as a global optimization method to determine the optimal weights in the priority metric. In order to evaluate the effectiveness of the proposed method, a traffic simulation model is developed in a high-fidelity traffic simulation environment called SUMO, based on a real-world traffic network. The traffic flow within this model is simulated using actual measured traffic data from the traffic network, enabling a comprehensive assessment of the novel optimal traffic signal control method in realistic conditions. The simulation results show that the proposed priority metric-based real-time traffic signal control algorithm can significantly reduce network-wide fuel consumption compared to the conventional fixed-time control and coordinated actuated control methods that are currently used in the modeled network. Additionally, incorporating truck priority in the priority metric leads to further improvements in fuel consumption reduction.

A Novel Deep Reinforcement Learning Approach to Traffic Signal Control with Connected Vehicles

The advent of connected vehicle (CV) technology offers new possibilities for a revolution in future transportation systems. With the availability of real-time traffic data from CVs, it is possible to more effectively optimize traffic signals to reduce congestion, increase fuel efficiency, and enhance road safety. The success of CV-based signal control depends on an accurate and computationally efficient model that accounts for the stochastic and nonlinear nature of the traffic flow. Without the necessity of prior knowledge of the traffic system’s model architecture, reinforcement learning (RL) is a promising tool to acquire the control policy through observing the transition of the traffic states. In this paper, we propose a novel data-driven traffic signal control method that leverages the latest in deep learning and reinforcement learning techniques. By incorporating a compressed representation of the traffic states, the proposed method overcomes the limitations of the existing methods in defining the action space to include more practical and flexible signal phases. The simulation results demonstrate the convergence and robust performance of the proposed method against several existing benchmark methods in terms of average vehicle speeds, queue length, wait time, and traffic density.

Traffic information interpolation method based on traffic flow emergence using swarm intelligence

Traffic congestion has become one of the most pressing social problems in today’s society, and research into appropriate traffic signal control is actively underway. At present, most traffic signal control methods define traffic signal parameters on the basis of traffic information such as the number of passing vehicles. Installing sensors at a vast number of intersections is necessary for more precise and real-time adaptive control, but this is unrealistic from the viewpoint of cost. As an alternative, we propose a swarm intelligence-based methodology that creates routes with a similar traffic volume using the traffic information from intersections already equipped with sensors and interpolates this information in the intersections without sensors in real time. Our simulation results show that the proposed methodology can effectively create similar traffic routes for main traffic flows with high traffic volumes. The results also show that it has an excellent interpolation performance for heavy traffic flows and can adapt and interpolate to situations where traffic flow changes suddenly. Moreover, the interpolation results are highly accurate at a road link where traffic flows confluence. We also developed an interpolation algorithm that is adaptable to traffic patterns with confluence traffic flows. Experiments were conducted with a simulation of merging traffic flows and the proposed method showed good results.

Cooperative Traffic Signal Control based on Biased ReLU Neural Network Approximation

Traffic signal control is important in intelligent transportation system, of which the cooperative control is difficult to realize but yet vital. Popular methods for solving this problem are based on multi-agent reinforcement learning (RL), in which function approximator, e.g., different kinds of neural network play a critical role. In this paper, we propose a multi-agent actor-critic RL framework with global value function and local policy function, for which the piecewise linear neural network, named biased ReLU (BReLU) is used as the function approximator. The reason for doing this is two-fold. First, it has been proved in the control literature that minimizing (maximizing) a piecewise linear function over a polyhedron yields piecewise linear solutions. Second, the BReLU neural network can provide a more accurate approximation than the traditional ReLU neural network when they have similar network structures. The proposed method is evaluated on the Simulation of Urban Mobility (SUMO) environment compared with two benchmark traffic signal control methods. The simulated results illustrate the proposed algorithm can coordinate the signal control between different intersections, achieve lower and more sustainable intersection delays on the whole traffic network.

Connected Transit Bus Dynamic Priority Weight Modeling and Conflicting Request Resolution Control at the Signalized Intersection

Multiaccess edge computing (MEC) and connected vehicle (CV) technologies have shown great potential and strength for traffic perception and real-time computing, which can be applied to enhance the efficiency of connected transit bus operations under their lower penetration conditions. Moreover, for the transit signal priority system, how to establish a model to measure traffic demand for conflicting priority request resolution and improve system response time has been widely researched for the last few decades. This paper proposes a dynamic priority weight (DPW) model for connected transit buses and a traffic signal control approach to coordinate multidirectional conflicting priority requests at a signalized intersection. The proposed model takes advantage of vehicle location, speed, and signal timing data to build time to change (TTOC) correlation functions to measure priority weights of both single-vehicle and directionality accumulation with consideration of vehicles arriving during the current green phase and conflict phase conditions; then, the aggregated priority weight value of each movement can be calculated in real-time. Once the maximum aggregated priority weight value among all movements is determined, the corresponding phase switch strategy is presented for the conflicting request resolution control problem. Homologous algorithm software for distributed deployment can be subsequently used for swift response. Simulation results show that the proposed DPW model-based traffic signal control method shows significant performance advancement, where the queueing vehicle number decrease exceeds 1 pcu/s and the throughput rate of major movements increases by approximately 2% without sacrificing the performance of minor movements in a large amount. What is more, it shows better delay optimization for social vehicles than the algorithm with delay as the objective while declining bus delay appreciable quantity with 43.4 s in average. Field test results also show that this method has excellent abilities to improve intersectional traffic capacity, for which queueing vehicle number and throughput rate indicators of all phases dramatically improved with 1.92 pcu/s and 6.68% on average, except for a slight degradation of individual minor traffic movements with 0.99 pcu/s and 0.11%.

Deep Reinforcement Learning for Intersection Signal Control Considering Pedestrian Behavior

Using deep reinforcement learning to solve traffic signal control problems is a research hotspot in the intelligent transportation field. Researchers have recently proposed various solutions based on deep reinforcement learning methods for intelligent transportation problems. However, most signal control optimization takes the maximization of traffic capacity as the optimization goal, ignoring the concerns of pedestrians at intersections. To address this issue, we propose a pedestrian-considered deep reinforcement learning traffic signal control method. The method combines a reinforcement learning network and traffic signal control strategy with traffic efficiency and safety aspects. At the same time, the waiting time of pedestrians and vehicles passing through the intersection is considered, and the Discrete Traffic State Encoding (DTSE) method is applied and improved to define the more comprehensive states and rewards. In the training of the neural network, the multi-process operation method is adopted, and multiple environments are run for training simultaneously to improve the model’s training efficiency. Finally, extensive simulation experiments are conducted on actual intersection scenarios using the simulation software Simulation of Urban Mobility (SUMO). The results show that compared to Dueling DQN, the waiting time due to our method decreased by 58.76% and the number of people waiting decreased by 51.54%. The proposed method can reduce both the number of people waiting and the waiting time at intersections.