Traffic Signal Control Research Articles

KeyLight: Intelligent Traffic Signal Control Method Based on Improved Graph Neural Network

Graph neural network combined with reinforcement learning is one of the most effective traffic signal control methods. However, existing methods fail to pay enough attention to the key information, such as the traffic information of the downstream section in extracting state and the intersection’s own state in aggregating information from neighbor intersections. As a result, adverse reactions like unstable learning and limited performance occur frequently when agents and models focus on interfering information and useless information too much. In this article, we propose KeyLight, an intelligent traffic signal control method based on reinforcement learning by facilitating the attention of the learning algorithm and model to the key information that is usually ignored. In KeyLight, we design a new state representation NOV-LADLE and introduce residual connection in the graph neural network to highlight the importance of the intersection’s state. Experiments show that, in the case of comparable throughput, the proposed KeyLight has been greatly improved and enhanced in performance. Especially, the average travel time can be increased by up to 23.44% on the real-world dataset.

An Architecture and Review of Intelligence Based Traffic Control System for Smart Cities

City traffic congestion can be reduced with the help of adaptable traffic signal control system. The technique improves the efficiency of traffic operations on urban road networks by quickly adjusting the timing of signal values to account for seasonal variations and brief turns in traffic demand. This study looks into how adaptive signal control systems have evolved over time, their technical features, the state of adaptive control research today, and Control solutions for diverse traffic flows composed of linked and autonomous vehicles. This paper finally came to the conclusion that the ability of smart cities to generate vast volumes of information, Artificial Intelligence (AI) approaches that have recently been developed are of interest because they have the power to transform unstructured data into meaningful information to support decision-making (For instance, using current traffic information to adjust traffic lights based on actual traffic circumstances). It will demand a lot of processing power and is not easy to construct these AI applications. Unique computer hardware/technologies are required since some smart city applications require quick responses. In order to achieve the greatest energy savings and QoS, it focuses on the deployment of virtual machines in software-defined data centers. Review of the accuracy vs. latency trade-off for deep learning-based service decisions regarding offloading while providing the best QoS at the edge using compression techniques. During the past, computationally demanding tasks have been handled by cloud computing infrastructures. A promising computer infrastructure is already available and thanks to the new edge computing advancement, which is capable of meeting the needs of tomorrow's smart cities.

Elevating adaptive traffic signal control in semi‐autonomous traffic dynamics by using connected and automated vehicles as probes

AbstractIn this work, the connected vehicle's messages are used to create an enhanced adaptive traffic signal control (ATSC) system for improved traffic flow. Few existing studies use connected and automated vehicles (CAVs) to develop traffic signal control algorithms under hybrid connected and autonomous conditions. The proposed approach focuses on a four‐phase traffic intersection with both CAVs and human‐driven vehicles (HVs). CAVs share real‐time state information, and a model called Roads Dynamic Segmentation estimates queuing procedures and vehicle fleet numbers on dynamic road sections. This information is used in the Store and Forward Model (SFM) to predict intersection queuing length. The ATSC system, based on model predictive control (MPC), aims to minimize intersection queue length while considering traffic constraints (undersaturated, saturated, and oversaturated) and avoiding free‐flow problems due to queue overflow. To reduce computational complexity, a linear‐quadratic‐regulator (LQR) is used. Real‐world vehicle trajectories and the SUMO tool are used for experimental purposes. Results show that the proposed method reduces average delay by 38.50% and 33.42% compared to fixed timing and traditional MPC in cases of oversaturated traffic flow with 100% CAV penetration. Even with a penetration rate of only 20%, average delay decreases by 13.65% and 6.50%, respectively. This study showcases not only the potential benefits of CAV in enhancing traffic, but also enables the optimal utilization of green duration in signalized intersection control systems. This helps prevent traffic congestion and ensures the efficient and smooth movement of traffic flow.

Enhancing the Robustness of Traffic Signal Control with StageLight: A Multiscale Learning Approach

The continuous evolution of artificial intelligence and cyber–physical systems has presented promising opportunities for optimizing traffic signal control in densely populated urban areas, with the aim of alleviating traffic congestion. One area that has garnered significant interest from both researchers and practitioners is the application of deep reinforcement learning (DRL) in traffic signal control. However, DRL-based algorithms often suffer from instability due to the dynamic nature of traffic flows. Discrepancies between the environments used for training and those encountered during deployment often lead to operational failures. Moreover, conventional DRL-based traffic signal control algorithms tend to reveal vulnerabilities when faced with unforeseen events, such as sensor failure. These challenges highlight the need for innovative solutions to enhance the robustness and adaptability of such systems. To address these pertinent issues, this paper introduces StageLight, a novel two-stage multiscale learning approach, which involves learning optimal timings on a coarse time scale in stage 1, while finetuning them on a finer time scale in stage 2. Our experimental results demonstrate StageLight’s remarkable capability to generalize across diverse traffic conditions and its robustness to various sensor-failure scenarios.

A Review of Deep Reinforcement Learning for Traffic Signal Control

Traffic signal control plays a vital role in effectively managing traffic flow and alleviating congestion in urban areas. Traditional methods for controlling traffic signals often rely on fixed timing plans or predefined algorithms, which may not be adaptable to changing traffic conditions. Reinforcement Learning is gaining traction as a favored data-centric method for adapting traffic signal control in intricate urban traffic networks. This article represents a conceptual review of recent studies and techniques that showcase the effectiveness of Deep Reinforcement Learning (DRL) in enhancing the performance of traffic signal control. These improvements include reducing travel time, fuel consumption, and emissions. Additionally, we will delve into different algorithms and learning systems explored in research papers, such as multi-agent reinforcement learning and Deep Q Networks (DQN).

Evaluating Signal Preemption Requests in Utah Using Vehicle-to-Everything Dedicated Short-Range Communication Equipped Snowplows

At the start of the 2019 to 2020 snow season, vehicle-to-everything (V2X) systems using dedicated short-range communication (DSRC) were placed on Utah Department of Transportation snowplows and traffic signal controllers on selected state routes. This study was conducted to understand the overall impacts of snowplows using V2X DSRC to request signal preemption. Roadside units (RSUs) were deployed on five corridors throughout the Salt Lake metropolitan area. Similar routes without RSUs were selected as a control and were used in the analysis to quantify results. Each snowplow on these corridors was equipped with an onboard unit (OBU). Based on data collected, analysis was performed on both traffic signal performance and vehicle performance data. Within the traffic signal performance analysis, it was found that the V2X DSRC system was utilized often, with snowplows requesting preemption in more 50% of the occasions they approached a signalized intersection. Of those requests, signal controllers granted preemption in over 80% of cases. On average, signal controller coordination was affected for less than 5 min. The vehicle performance analysis found that the snowplows on equipped routes had travel speeds that were less affected when there was snow than on corresponding not-equipped routes. Vehicle crash data also showed that there was a greater decrease in crashes on equipped routes than not-equipped routes. Anecdotal evidence gathered from snowplow drivers indicated that snowplows stopped less when using signal preemption. Drivers also noted a benefit to overall snow removal operations on corridors equipped with the V2X DSRC system.

Optimization Control of Adaptive Traffic Signal with Deep Reinforcement Learning

The optimization and control of traffic signals is very important for logistics transportation. It not only improves the operational efficiency and safety of road traffic, but also conforms to the direction of the intelligent, green, and sustainable development of modern cities. In order to improve the optimization effect of traffic signal control, this paper proposes a traffic signal optimization method based on deep reinforcement learning and Simulation of Urban Mobility (SUMO) software for urban traffic scenarios. The intersection training scenario was established using SUMO micro traffic simulation software, and the maximum vehicle queue length and vehicle queue time were selected as performance evaluation indicators. In order to be more relevant to the real environment, the experiment uses Weibull distribution to simulate vehicle generation. Since deep reinforcement learning takes into account both perceptual and decision-making capabilities, this study proposes a traffic signal optimization control model based on the deep reinforcement learning Deep Q Network (DQN) algorithm by considering the realism and complexity of traffic intersections, and first uses the DQN algorithm to train the model in a training scenario. After that, the G-DQN (Grouping-DQN) algorithm is proposed to address the problems that the definition of states in existing studies cannot accurately represent the traffic states and the slow convergence of neural networks. Finally, the performance of the G-DQN algorithm model was compared with the original DQN algorithm model and Advantage Actor-Critic (A2C) algorithm model. The experimental results show that the improved algorithm increased the main indicators in all aspects.

MARLens: Understanding Multi-Agent Reinforcement Learning for Traffic Signal Control Via Visual Analytics.

The issue of traffic congestion poses a significant obstacle to the development of global cities. One promising solution to tackle this problem is intelligent traffic signal control (TSC). Recently, TSC strategies leveraging reinforcement learning (RL) have garnered attention among researchers. However, the evaluation of these models has primarily relied on fixed metrics like reward and queue length. This limited evaluation approach provides only a narrow view of the model's decision-making process, impeding its practical implementation. Moreover, effective TSC necessitates coordinated actions across multiple intersections. Existing visual analysis solutions fall short when applied in multi-agent settings. In this study, we delve into the challenge of interpretability in multi-agent reinforcement learning (MARL), particularly within the context of TSC. We propose MARLens, a visual analytics system tailored to understand MARL-based TSC. Our system serves as a versatile platform for both RL and TSC researchers. It empowers them to explore the model's features from various perspectives, revealing its decision-making processes and shedding light on interactions among different agents. To facilitate quick identification of critical states, we have devised multiple visualization views, complemented by a traffic simulation module that allows users to replay specific training scenarios. To validate the utility of our proposed system, we present three comprehensive case studies, incorporate insights from domain experts through interviews, and conduct a user study. These collective efforts underscore the feasibility and effectiveness of MARLens in enhancing our understanding of MARL-based TSC systems and pave the way for more informed and efficient traffic management strategies.

Reinforcement Learning Based Urban Traffic Signal Control and Its Impact Assessment on Environmental Pollution

To address the growing complexity of urban traffic congestion and its associated environmental impacts, this study presents a pioneering application of the Gaussian plume model to investigate the carbon dioxide emission reduction efficacy of various reinforcement learning algorithms within a traffic signal control framework. By employing an insightful fusion of the traditional environmental science tool with contemporary reinforcement learning strategies - specifically Independent Partially Observable Policy Optimization (IPPO), Independent Delay Q-Network (IDQN), and MPLight - this research marks a novel intersection of methodologies. By quantitatively simulating and analyzing the diffusion dynamics of carbon dioxide pollutants under different traffic signal control scenarios, the study not only highlights the innovative use of the Gaussian plume model to assess the environmental impact of traffic signal control, but also provides critical insights into the selection and optimization of traffic signal control algorithms for improved urban environmental sustainability.

Ant Colony Optimization With Look-Ahead Mechanism for Dynamic Traffic Signal Control of IoV Systems

In the era of Internet of Vehicle (IoV), traffic data are obtained by wireless sensor networks (WSNs), which provides convenience for real-time traffic signal control and is expected to further improve the efficiency of the whole system. Traffic signal controllers designed for IoV systems usually fully depend on data transmitted from the IoV system and are limited to the penetration rate of connected vehicles. This work proposes a dynamic ant colony optimization algorithm with the look-ahead mechanism (DALM) with eight no-conflict phases. The needed traffic information in the DALM algorithm is small and can be obtained independently by the system, which makes this method does not fully rely on the vehicle trajectories data and can better protect customer privacy and deal with cyber attack. With the look-ahead mechanism, the forthcoming traffic flow together with the current waiting queue are involved to predict the length of waiting vehicles and the optimal phase sequence is dynamically selected according to shortest green-light duration of candidate phase sequences obtained form the collected real-time traffic data. Experiments conducted under five scenarios with different kinds of traffic flow on SUMO platform show that compared with other four methods, both the total waiting time and the total carbon emissions of vehicles at the intersection are greatly reduced by using the DALM algorithm, especially in high traffic flow situations. The proposed algorithm can greatly improve the efficiency of the intersection in high traffic flow situation and is predictable to benefit the real-time traffic signal control in IoV system.

Knowledge transfer in multi-objective multi-agent reinforcement learning via generalized policy improvement

Even though many real-world problems are inherently distributed and multi-objective, most of the reinforcement learning (RL) literature deals with single agents and single objectives. While some of these problems can be solved using a single-agent single-objective RL solution (e.g., by specifying preferences over objectives), there are robustness issues, as well the fact that preferences may change over time, or it might not even be possible to set such preferences. Therefore, a need arises for a way to train multiple agents for any given preference distribution over the objectives. This work thus proposes a multi-objective multi-agent reinforcement learning (MOMARL) method in which agents build a shared set of policies during training, in a decentralized way, and then combine these policies using a generalization of policy improvement and policy evaluation (fundamental operations of RL algorithms) to generate effective behaviors for any possible preference distribution, without requiring any additional training. This method is applied to two different application scenarios: a multi-agent extension of a domain commonly used in the related literature, and traffic signal control, which is more complex, inherently distributed and multi-objective (the flow of both vehicles and pedestrians are considered). Results show that the approach is able to effectively and efficiently generate behaviors for the agents, given any preference over the objectives.

Efficient centralized traffic grid signal control based on meta-reinforcement learning

Dear Editor, This letter presents a novel method to tackle the two challenges of the centralized traffic control based on reinforcement learning (RL): the curse of dimensionality as the scale of traffic grid increases and the data-inefficiency problem that requires large amounts of samples to learn. First, we use a sequence-to-sequence (seq2seq) model and the attention mechanism to decompose the state-action space into sub-spaces, thus dealing with the first challenge. Second, we propose a new context-based meta-RL model that disentangles task inference and control, which improves the meta-training efficiency and accelerates the learning process in the new environment. We evaluate our approach on real-world datasets and the results demonstrate that our approach outperforms the state-of-the-art deep reinforcement learning (DRL)-based methods and the traditional control methods.

Scalable Reinforcement Learning Framework for Traffic Signal Control Under Communication Delays

Scalable Reinforcement Learning Framework for Traffic Signal Control Under Communication Delays

Large-Scale Traffic Signal Control Based on Multi-Agent Q-Learning and Pressure

Large-Scale Traffic Signal Control Based on Multi-Agent Q-Learning and Pressure

Unlocking Traffic Control: Exploring Intersection Dynamics with Visible Light Communication

This paper introduces the use of Visible Light Communication (VLC) to enhance traffic signal efficiency and vehicle trajectory management at urban intersections. By integrating VLC localization with learning-based traffic signal control, a multi-intersection traffic system is proposed. VLC enables communication between connected vehicles and infrastructure through mobile optical receivers. The primary objectives are to reduce waiting times and improve overall traffic safety by accommodating diverse traffic movements during multiple signal phases. Cooperative mechanisms and queue/response interactions balance traffic flow between intersections, enhancing road network performance. A reinforcement learning scheme optimally schedules traffic signals, with agents at each intersection using VLC-enabled vehicle communication to improve traffic flow and overall system optimization. Evaluated using the SUMO urban mobility simulator, the system demonstrates reduced waiting and travel times. The decentralized and scalable nature of this approach highlights its potential applicability in real-world traffic scenarios.

Enhancing Urban Traffic Management with Visible Light Communication and Reinforcement Learning

This paper introduces Visible Light Communication (VLC) to enhance traffic signal efficiency and vehicle trajectory management at urban intersections. A multi-intersection traffic control system is proposed, integrating VLC localization services with learning-based traffic signal control. VLC facilitates communication between connected vehicles and infrastructure using headlights, streetlights, and traffic signals to transmit information. By leveraging vehicle-to-vehicle (V2V) and infrastructure-to-vehicle (I2V) interactions, joint transmission and data collection are achieved via mobile optical receivers. The system aims to reduce waiting times for pedestrians and vehicles while improving overall traffic safety. It is designed to be flexible and adaptive, accommodating diverse traffic movements during multiple signal phases. VLC cooperative mechanisms, transmission range, relative pose concepts, and queue/request/response interactions help balance traffic flow between intersections, enhancing the overall road network performance.

BlindLight: High Robustness Reinforcement Learning Method to Solve Partially Blinded Traffic Signal Control Problem

BlindLight: High Robustness Reinforcement Learning Method to Solve Partially Blinded Traffic Signal Control Problem

Model-Based Graph Reinforcement Learning for Inductive Traffic Signal Control

Model-Based Graph Reinforcement Learning for Inductive Traffic Signal Control

A Novel, Three-Stage Intelligent Fuzzy Traffic Signal Control System

A Novel, Three-Stage Intelligent Fuzzy Traffic Signal Control System

Prediction of Traffic Congestion in Vehicular Ad-Hoc Networks Employing Extreme Deep Learning Machines (Edrlm)

Vehicular Ad-Hoc Networks (VANETs) represent a crucial component of intelligent transportation systems (ITS), enabling vehicles to communicate with each other and with roadside infrastructure. Predicting traffic congestion in VANETs is essential for enhancing road safety, optimizing traffic flow, and improving overall transportation efficiency. Traditional machine learning methods have shown promise in this domain; however, they often fall short in handling the complex, high-dimensional data typical of VANETs. To address these challenges, this study employs Extreme Deep Learning Machines (EDRLM), an advanced deep learning technique, for traffic congestion prediction. The EDRLM framework leverages the strengths of deep neural networks and extreme learning machines, offering a robust and scalable solution for processing the dynamic and heterogeneous data in VANETs. By integrating feature extraction, selection, and prediction into a unified model, EDRLM can capture intricate patterns and temporal dependencies within traffic data. The proposed model is trained and validated using real-world VANET datasets, incorporating various traffic parameters such as vehicle speed, density, and inter-vehicular distances. Our experimental results demonstrate that EDRLM outperforms conventional machine learning algorithms in terms of prediction accuracy, computational efficiency, and robustness to noise and missing data. The model's ability to provide timely and precise congestion predictions can facilitate proactive traffic management strategies, including dynamic routing and adaptive traffic signal control, ultimately leading to reduced travel times and enhanced road safety. This study underscores the potential of EDRLM in transforming traffic management in VANETs, paving the way for more intelligent and adaptive ITS solutions. Future research directions include exploring hybrid models combining EDRLM with other advanced machine learning techniques and expanding the framework to accommodate emerging vehicular communication technologies such as 5G and Internet of Things (IoT) devices.