Traffic Signal Control Research Articles

Cooperative Traffic Signal Control Using a Distributed Agent-Based Deep Reinforcement Learning With Incentive Communication

Cooperative Traffic Signal Control Using a Distributed Agent-Based Deep Reinforcement Learning With Incentive Communication

Two‐stage algorithm for traffic signal optimization and web‐service system development

AbstractEfficient control of traffic signals for vehicles and pedestrians at intersections is critical for relieving traffic congestion. Considering the unique characteristics of intersections, such as the number of roads, the presence or absence of crosswalks, road geometric shapes, and traffic demand patterns, an appropriate phase sequence and duration for traffic signals must be established at each intersection. This paper proposes a simulation‐based two‐stage algorithm comprising integer‐constrained Adam (ICA) and tabu search (TS) to optimize the phase sequence and duration for arbitrary intersections with arbitrary traffic‐demand patterns. The ICA promptly identifies a promising region in which a global optimal solution is likely to be obtained, whereas TS determines the best solution near the region. The performance of the proposed algorithm that optimizes phase durations with fixed phase sequence is evaluated against several baseline methods using 24 instances across six actual intersections. Experimental results show that the proposed algorithm reduces the average travel time by 20.2% compared with existing traffic signals within a computation time of 4 min, thus providing a near‐optimal solution eight times faster than commonly used population‐based metaheuristics. Furthermore, the algorithm demonstrates robust performance across heterogeneous vehicles and recommends the best phase sequence that effectively alleviates congestion in current traffic signal systems. The optimized phase sequence with best phase durations further reduces the average travel time by approximately 11.3% compared with the existing phase sequence with best phase durations at an actual intersection. To facilitate its widespread use, a free, open web‐service system named “Smart Intersection for Traffic Efficiency” is developed, which enables users to optimize traffic signal systems without requiring optimization background or simulation knowledge.

Integration Traffic Signal Control From Synchro to SUMO

This study investigates the feasibility and challenges of transferring traffic signal control schemes from the macroscopic signal timing optimization tool Synchro to the microscopic traffic simulator SUMO, focusing on Downtown Seattle as a case study. The research assesses the process of sharing and importing traffic signal timing plans, a crucial aspect of transportation simulations, between these two platforms. We conduct a detailed analysis of the traffic signal characteristics and data formats unique to each simulator and identify elements suitable for conversion. Subsequently, a four-stage framework is developed for semi-automatic integration of traffic signal control between the two. Our results indicate a successful conversion rate of approximately 85% of signalized intersections from Synchro to SUMO. This research not only illustrates the challenges and solutions in converting signal control across different platforms but also paves the way for future studies aimed at improving the interoperability of various traffic simulation tools.

A survey on reinforcement learning-based control for signalized intersections with connected automated vehicles

ABSTRACT Recent advancements in connected automated vehicles (CAVs) and reinforcement learning (RL) hold significant promise for enhancing intelligent traffic control systems. This paper conducts a systematic review of studies on RL-based urban traffic control at signalised intersections, highlighting the significant impact of CAVs on traffic control performance improvement. We first review the fundamental concepts of RL algorithms, establishing a foundational understanding for subsequent RL-based traffic control methods. We then review recent progress in RL-based traffic signal control using CV/CAV trajectory data, RL-based CAV trajectory planning, and the cooperative control of both traffic signals and CAVs at signalised intersections. Our aim is to provide researchers with a comprehensive roadmap for future research in RL-based traffic control at signalised intersections.

Systematic approach to a government‐led technology roadmap for future‐ready adaptive traffic signal control systems

AbstractThe economic impact of inefficient traffic control systems is significant owing to prolonged commute durations, and increased energy consumption. Traffic signal control systems (TSCSs) significantly influence traffic flows at intersections. Therefore, adaptive TSCSs (ATSCSs) that can adjust to traffic conditions in real‐time have been proposed as more efficient alternatives. However, the expensive implementation of these systems highlights the need for judicious investments in appropriate technologies and infrastructure. Therefore, a comprehensive technology roadmap should be built that guides the future development of traffic control and the infusion of technologies to address traffic needs. Additionally, as ATSCSs are developed and managed by local governments, the perspective of a government‐led technology roadmap is required to guide the roadmap development and implementation. Although studies have explored technology roadmaps across numerous sectors, the viewpoint of roadmap development guided by government entities is frequently neglected despite the role of these entities in shaping technological policies, underwriting research and development initiatives, and driving nationwide innovation strategies. In this study, a comprehensive framework is proposed for developing technology roadmaps tailored for systems and technologies led by governmental entities. This framework has been adapted from the Advanced Technology Roadmap Architecture (ATRA) and brought original adjustments thereby addressing the research gap. The study also presents strategic recommendations for the ATSCS implementation in South Korea, integrating systems engineering principles for a holistic approach to technological advancements. The framework can be replicated to serve as a guide for governments seeking to implement effective and efficient technology roadmaps for public infrastructure systems.

Enhancing Pedestrian Safety by Providing a LiDAR-Based Analysis of Jaywalking Conflicts at Signalized Intersections

Motives: In response to the inherent vulnerability of pedestrians in urban settings, this paper is driven by a commitment to enhancing their mobility and safety. Recognizing the prevalence of jaywalking as a significant concern, the study seeks practical solutions through the application of LiDAR sensors at signalized intersections. By delving into the complexities of jaywalking events and their contributing factors, the research aims to provide valuable insights that extend beyond mere statistical analysis. The motivations behind this endeavour lie in the imperative to comprehensively understand and address the risks associated with jaywalking, ultimately fostering a safer environment for pedestrians navigating urban crossroads. Aim: The primary aim of this paper is to assess and analyse the diverse factors influencing the frequency of jaywalking at signalized intersections, leveraging the capabilities of LiDAR sensors for safety applications. Through a meticulous examination of 1000 jaywalking events detected over a six-month period, the study aims to pinpoint highly correlated independent variables to the frequency of jaywalking events. These variables include traffic signal controller patterns, signal phases, vehicle-pedestrian conflicts, weather conditions, vehicle volume, walking patterns toward the median, pedestrian volume, and the unique jaywalker’s ratio. Employing advanced statistical regression models, the research seeks to identify optimal models and unravel key insights into the nuanced dynamics of jaywalking behaviour. The overarching goal is to equip decision-makers and transportation specialists with data-driven knowledge, enabling them to implement targeted safety measures that mitigate pedestrian risks and enhance safety infrastructure at critical urban crossroads. Results: The outcomes of the study, derived from the optimal Poisson regression model, yield crucial insights into the multifaceted nature of jaywalking events at signalized intersections. The morning and mid-day signal controller patterns exhibit a substantial decrease of 44.7% and 34.4%, respectively, compared to the evening (PM) pattern, shedding light on temporal nuances in jaywalking behaviour. Additionally, the severity of vehicle-pedestrian conflicts escalates proportionally with the number of jaywalkers, emphasizing the importance of addressing pedestrian flow in mitigating potential conflicts. Notably, the presence of vegetation in the median emerges as a significant factor, significantly increasing the frequency of jaywalking. These results contribute to a nuanced understanding of the intricate interplay between environmental, temporal, and behavioural factors in jaywalking incidents. Decision-makers and transportation specialists can leverage these findings to formulate targeted safety interventions, fostering a safer pedestrian experience at crucial urban crossroads

Optimizing green splits in high‐dimensional traffic signal control with trust region Bayesian optimization

AbstractCentralized traffic signal control has long been a challenging, high‐dimensional optimization problem. This study establishes a simulation‐based optimization framework and develops a novel optimization algorithm based on trust region Bayesian optimization (TuRBO), which can efficiently obtain an approximate optimal solution to the high‐dimensional traffic signal control problem. Local Gaussian process (GP), trust region, and Thompson sampling are employed in the TuRBO and contribute considerably to performance in terms of computational speed, solution quality, and scalability. Empirical studies are carried out using data from Mudanjiang and Chengdu, China. The performance of TuRBO is compared with that of Bayesian optimization (BO), genetic algorithm and random sampling. The results show that TuRBO converges the fastest because of its ability to balance exploration and exploitation through the trust region and Thompson sampling. Meanwhile, because TuRBO enables more efficient exploitation through the local GP, the solution quality of TuRBO outperforms others significantly. The average waiting time achieved by TuRBO was 2.84% lower than that achieved by BO. Finally, the method has been successfully extended to a large network with 233‐dimensional spaces and 122 signalized intersections, demonstrating that the developed methodology can deal with high‐dimensional traffic signal control effectively for real case applications.

Traffic Signal Control Optimization Based on Neural Network in the Framework of Model Predictive Control

To improve the effectiveness of model predictive control (MPC) in dynamic traffic signal control strategies, it has been combined with graph convolutional networks (GCNs) and deep reinforcement learning (DRL) technologies. In this study, a neural-network-based traffic signal control optimization method under the MPC framework is proposed. A dynamic correlation matrix is introduced in the predictive model to adapt to the dynamic changes in correlations between nodes over time. The signal control optimization strategy is solved using DRL, where the agent explores the optimal control strategy based on pre-set constraints in the future road environment. The geometric structure and traffic flow data of a real intersection were selected as the simulation validation environment, and a joint simulation was conducted using Python and SUMO. The experimental results indicate that in low-traffic scenarios, the queue length is reduced by more than 2 vehicles compared to the selected comparison methods; in high-traffic scenarios, the queue length is reduced by an average of 17 vehicles. Under the actual traffic data of the intersection, the average speed is increased by 6.4% compared to the fixed timing method; compared to the inductive signal control method, it increases from 9.76 m/s to 11.69 m/s, an improvement of 19.7%, effectively enhancing the intersection signal control performance.

Large-Scale Traffic Signal Control Using Constrained Network Partition and Adaptive Deep Reinforcement Learning

Large-Scale Traffic Signal Control Using Constrained Network Partition and Adaptive Deep Reinforcement Learning

Intersec2vec-TSC: Intersection Representation Learning for Large-Scale Traffic Signal Control

Intersec2vec-TSC: Intersection Representation Learning for Large-Scale Traffic Signal Control

Dual-Objective Reinforcement Learning-Based Adaptive Traffic Signal Control for Decarbonization and Efficiency Optimization

To improve traffic efficiency, adaptive traffic signal control (ATSC) systems have been widely developed. However, few studies have proactively optimized the air environmental issues in the development of ATSC. To fill this research gap, this study proposes an optimized ATSC algorithm to take into consideration both traffic efficiency and decarbonization. The proposed algorithm is developed based on the deep reinforcement learning (DRL) framework with dual goals (DRL-DG) for traffic control system optimization. A novel network structure combining Convolutional Neural Networks and Long Short-Term Memory Networks is designed to map the intersection traffic state to a Q-value, accelerating the learning process. The reward mechanism involves a multi-objective optimization function, employing the entropy weight method to balance the weights among dual goals. Based on a representative intersection in Changsha, Hunan Province, China, a simulated intersection scenario is constructed to train and test the proposed algorithm. The result shows that the ATSC system optimized by the proposed DRL-DG results in a reduction of more than 71% in vehicle waiting time and 46% in carbon emissions compared to traditional traffic signal control systems. It converges faster and achieves a balanced dual-objective optimization compared to the prevailing DRL-based ATSC.

Smart Control of Traffic Light Using Image detection

Abstract: Urban areas are increasingly grappling with the issue of traffic congestion, a problem exacerbated by growing populations and the proliferation of motor vehicles. This not only leads to delays and increased stress for commuters, but also contributes to greater fuel usage and environmental pollution. This issue is particularly pronounced in large metropolitan areas. The escalating nature of this problem underscores the necessity for real-time assessments of road traffic density, which can lead to more effective traffic management strategies and signal control. The role of the traffic man controller is pivotal in influencing the flow of traffic, necessitating the optimization of traffic management system to cater to this burgeoning demand. Our proposed solution leverages live footage from intersection cameras to estimate traffic density through image processing and AI techniques. The system also incorporates a traffic signal switching algorithm based on vehicle density, aiming to alleviate traffic congestion. This, in turn, facilitates smoother movement for commuters and helps to mitigate pollution

Constrained Reinforcement Learning for Fair and Environmentally Efficient Traffic Signal Controllers

Traffic signal controller (TSC) has a crucial role in managing traffic flow in urban areas. Recently, reinforcement learning models have received a great attention for TSC with promising results. However, these RL-TSC models still need to be improved for real-world deployment due to limited exploration of different performance metrics such as fair traffic scheduling or air quality impact. In this work, we introduce a constrained multi-objective RL model that minimizes multiple constrained objectives while achieving a higher expected reward. Furthermore, our proposed RL strategy integrates the peak and average constraint models to the RL problem formulation with maximum entropy off-policy models. We applied this strategy to a single TSC and a network of TSCs. As part of this constrained RL-TSC formulation, we discuss fairness and air quality parameters as constraints for the close-loop control system optimization model at TSCs called FAirLight . Our experimental analysis shows that the proposed FAirLight achieves a good traffic flow performance in terms of average waiting time while being fair and environmentally friendly. Our method outperforms the baseline models and allows a more comprehensive view of RL-TSC regarding its applicability to the real world.

E-DBRL: efficient double broad reinforcement learning for adaptive traffic signal control

E-DBRL: efficient double broad reinforcement learning for adaptive traffic signal control

Digital Twin-Enhanced Adaptive Traffic Signal Framework under Limited Synchronization Conditions

Unmanned traffic signal control is regarded as a sustainable intelligent management methodology. However, it faces the challenge of unpredictable traffic flow due to stochastic arrivals. The digital twin (DT) has emerged as a promising approach to address the challenges of time-varying traffic demand in urban transportation. Previous studies of DT-based adaptive traffic signal control (ATSC) methods all assume ideal synchronization conditions between the DT and the physical twin (PT). It means that the DT can immediately figure out the next neglecting limitation of realistic conditions, i.e., discrepancies between the DT and PT and computational ability. This paper proposes a DT-ATSC framework aimed at reducing the total delay at isolated intersections under limited synchronization conditions. The framework contains two parts: (1) a cell transmission model-based intersection simulation model featuring less computational consumption and the parameter self-calibration mechanism; and (2) scheme searching algorithms that can guide the DT to create optimization-oriented signal timing scheme candidates. Three options are provided for the scheme searching algorithms, i.e., grid search (GS), the genetic algorithm (GA), and Bayesian optimization (BO). A testing platform is created to validate the effectiveness of the proposed DT-ATSC. Experimental results indicate that the proposed DT-ATSC-BO outperforms the DT-ATSC-GA and DT-ATSC-GS. Meanwhile, the average vehicle delay of the DT-ATSC-BO is up to 53% lower than that of the current adaptive signal control method, which indicates that the proposed DT-ATSC has achieved the expected effect. Moreover, compared to the previous related work, the proposed DT-ATSC framework is more likely to be able to be applied in realistic situations because synchronization issues are incorporated in the design of the DT-ATSC by assuming a limited margin time for a decision.

First steps towards real-world traffic signal control optimisation by reinforcement learning

ABSTRACT Enhancing traffic signal optimisation has the potential to improve urban traffic flow without the need for expensive infrastructure modifications. While reinforcement learning (RL) techniques have demonstrated their effectiveness in simulations, their real-world implementation is still a challenge. Real-world systems need to be developed that guarantee a deployable action definition for real traffic systems while prioritising safety constraints and robust policies. This paper introduces a method to overcome this challenge by introducing a novel action definition that optimises parameter-level control programmes designed by traffic engineers. The complete proposed framework consists of a traffic situation estimation, a feature extractor, and a system that enables training on estimates of real-world traffic situations. Further multimodal optimisation, scalability, and continuous training after deployment could be achieved. The first simulative tests using this action definition show an average improvement of more than 20% in traffic flow compared to the baseline – the corresponding pre-optimised real-world control.

A multi‐agent deep reinforcement learning approach for traffic signal coordination

AbstractThe purpose of signal control is to allocate time for competing traffic flows to ensure safety. Artificial intelligence has made transportation researchers more interested in adaptive traffic signal control, and recent literature confirms that deep reinforcement learning (DRL) can be effectively applied to adaptive traffic signal control. Deep neural networks enhance the learning potential of reinforcement learning. This study applies the DRL method, Double Deep Q‐Network, to train local agents. Each local agent learns independently to accommodate the regional traffic flows and dynamics. After completing the learning, a global agent is created to integrate and unify the action policies selected by each local agent to achieve the purpose of traffic signal coordination. Traffic flow conditions are simulated through the simulation of urban mobility. The benefits of the proposed approach include improving the efficiency of intersections and minimizing the overall average waiting time of vehicles. The proposed multi‐agent reinforcement learning model significantly improves the average vehicle waiting time and queue length compared with the results from PASSER‐V and pre‐timed signal setting strategies.

CycLight: Learning traffic signal cooperation with a cycle-level strategy

This study introduces CycLight, a novel cycle-level deep reinforcement learning (RL) approach for network-level adaptive traffic signal control (NATSC) systems. Traditional RL-based step-by-step traffic signal controllers, while precise and responsive, demand high computational resources for frequent data exchanges and can increase traffic accident risks due to disordered phase switching. Different from the step-by-step control strategy aforementioned, the proposed CycLight adopts a cycle-level strategy, optimizing cycle length and splits simultaneously using Parameterized Deep Q-Networks (PDQN) algorithm. This cycle-level approach effectively reduces the computational burden associated with frequent data communication, meanwhile enhancing the practicality and safety of real-world applications. Moreover, a decentralized framework is formulated for multi-agent cooperation, while attention mechanism is integrated to accurately assess the impact of the surroundings on the current intersection. CycLight is tested in a large synthetic traffic grid using the microscopic traffic simulation tool, Simulation of Urban MObility (SUMO). Experimental results not only demonstrate the superiority of CycLight over other state-of-the-art approaches but also showcase its robustness against information transmission delays.

Adaptive urban traffic signal control based on enhanced deep reinforcement learning

One of the focal points in the field of intelligent transportation is the intelligent control of traffic signals (TS), aimed at enhancing the efficiency of urban road networks through specific algorithms. Deep Reinforcement Learning (DRL) algorithms have become mainstream, yet they suffer from inefficient training sample selection, leading to slow convergence. Additionally, enhancing model robustness is crucial for adapting to diverse traffic conditions. Hence, this paper proposes an enhanced method for traffic signal control (TSC) based on DRL. This approach utilizes dueling network and double q-learning to alleviate the overestimation issue of DRL. Additionally, it introduces a priority sampling mechanism to enhance the utilization efficiency of samples in memory. Moreover, noise parameters are integrated into the neural network model during training to bolster its robustness. By representing high-dimensional real-time traffic information as matrices, and employing a phase-cycled action space to guide the decision-making of intelligent agents. Additionally, utilizing a reward function that closely mirrors real-world scenarios to guide model training. Experimental results demonstrate faster convergence and optimal performance in metrics such as queue length and waiting time. Testing experiments further validate the method's robustness across different traffic flow scenarios.

Integrating vehicle trajectory planning and arterial traffic management to facilitate eco-approach and departure deployment

Eco-approach and departure (EAD) enable continuous vehicle motion in urban signalized corridors. Since such a motion can extend to the EAD vehicles’ followers, it makes EAD a promising technology to benefit the traffic flow where automated vehicles and conventional vehicles coexist. Most existing EAD studies envision an ideal setting that neglects real-world operational conditions such as lane changes, multi-movement intersection configuration, partially automated fleet, and/or limited traffic state awareness. This study aims to fill the gap by designing an EAD algorithm considering real-world traffic operation constraints. The proposed algorithm uses a model predictive controller to minimize vehicle speed reduction and variation based on the real-time traffic signal control plan and measured queues at the intersection. The required inputs are readily available at many modern intersections. We observed that the proposed controller’s performance might degrade because of lane-changing maneuvers and lead-left turn traffic signals. These observations motivated our development of a lane change management strategy and a signal control implementation strategy to facilitate the EAD implementation. The lane change management strategies separate the EAD operations and lane-changing maneuvers in time and space. The signal control implementation strategy applies lag-left turn signals to enable EAD operation for both the through and left-turn vehicles. Compared to the non-EAD case, our EAD approach produces 2.5% to 7.8% energy savings while keeping similar intersection mobility. Notably, this approach brings about 2.5% to 3.6% energy savings in a 2% CAV case. This result demonstrates the feasibility of deploying EAD at low connected automated vehicle penetration rates.