Adaptive Traffic Signal Control Research Articles

Systematic approach to a government‐led technology roadmap for future‐ready adaptive traffic signal control systems

Systematic approach to a government‐led technology roadmap for future‐ready adaptive traffic signal control systems

E-DBRL: efficient double broad reinforcement learning for adaptive traffic signal control

E-DBRL: efficient double broad reinforcement learning for adaptive traffic signal control

Digital Twin-Enhanced Adaptive Traffic Signal Framework under Limited Synchronization Conditions

Unmanned traffic signal control is regarded as a sustainable intelligent management methodology. However, it faces the challenge of unpredictable traffic flow due to stochastic arrivals. The digital twin (DT) has emerged as a promising approach to address the challenges of time-varying traffic demand in urban transportation. Previous studies of DT-based adaptive traffic signal control (ATSC) methods all assume ideal synchronization conditions between the DT and the physical twin (PT). It means that the DT can immediately figure out the next neglecting limitation of realistic conditions, i.e., discrepancies between the DT and PT and computational ability. This paper proposes a DT-ATSC framework aimed at reducing the total delay at isolated intersections under limited synchronization conditions. The framework contains two parts: (1) a cell transmission model-based intersection simulation model featuring less computational consumption and the parameter self-calibration mechanism; and (2) scheme searching algorithms that can guide the DT to create optimization-oriented signal timing scheme candidates. Three options are provided for the scheme searching algorithms, i.e., grid search (GS), the genetic algorithm (GA), and Bayesian optimization (BO). A testing platform is created to validate the effectiveness of the proposed DT-ATSC. Experimental results indicate that the proposed DT-ATSC-BO outperforms the DT-ATSC-GA and DT-ATSC-GS. Meanwhile, the average vehicle delay of the DT-ATSC-BO is up to 53% lower than that of the current adaptive signal control method, which indicates that the proposed DT-ATSC has achieved the expected effect. Moreover, compared to the previous related work, the proposed DT-ATSC framework is more likely to be able to be applied in realistic situations because synchronization issues are incorporated in the design of the DT-ATSC by assuming a limited margin time for a decision.

Digital Twin-Enhanced Adaptive Traffic Signal Framework under Limited Synchronization Conditions

Unmanned traffic signal control is regarded as a sustainable intelligent management methodology. However, it faces the challenge of unpredictable traffic flow due to stochastic arrivals. The digital twin (DT) has emerged as a promising approach to address the challenges of time-varying traffic demand in urban transportation. Previous studies of DT-based adaptive traffic signal control (ATSC) methods all assume ideal synchronization conditions between the DT and the physical twin (PT). It means that the DT can immediately figure out the next neglecting limitation of realistic conditions, i.e., discrepancies between the DT and PT and computational ability. This paper proposes a DT-ATSC framework aimed at reducing the total delay at isolated intersections under limited synchronization conditions. The framework contains two parts: (1) a cell transmission model-based intersection simulation model featuring less computational consumption and the parameter self-calibration mechanism; and (2) scheme searching algorithms that can guide the DT to create optimization-oriented signal timing scheme candidates. Three options are provided for the scheme searching algorithms, i.e., grid search (GS), the genetic algorithm (GA), and Bayesian optimization (BO). A testing platform is created to validate the effectiveness of the proposed DT-ATSC. Experimental results indicate that the proposed DT-ATSC-BO outperforms the DT-ATSC-GA and DT-ATSC-GS. Meanwhile, the average vehicle delay of the DT-ATSC-BO is up to 53% lower than that of the current adaptive signal control method, which indicates that the proposed DT-ATSC has achieved the expected effect. Moreover, compared to the previous related work, the proposed DT-ATSC framework is more likely to be able to be applied in realistic situations because synchronization issues are incorporated in the design of the DT-ATSC by assuming a limited margin time for a decision.

Depthwise Convolution for Multi-Agent Communication With Enhanced Mean-Field Approximation.

Multi-Agent settings remain a fundamental challenge in the reinforcement learning (RL) domain due to the partial observability and the lack of accurate real-time interactions across agents. In this article, we propose a new method based on local communication learning to tackle the multi-agent RL (MARL) challenge within a large number of agents coexisting. First, we design a new communication protocol that exploits the ability of depthwise convolution to efficiently extract local relations and learn local communication between neighboring agents. To facilitate multi-agent coordination, we explicitly learn the effect of joint actions by taking the policies of neighboring agents as inputs. Second, we introduce the mean-field approximation into our method to reduce the scale of agent interactions. To more effectively coordinate behaviors of neighboring agents, we enhance the mean-field approximation by a supervised policy rectification network (PRN) for rectifying real-time agent interactions and by a learnable compensation term for correcting the approximation bias. The proposed method enables efficient coordination as well as outperforms several baseline approaches on the adaptive traffic signal control (ATSC) task and the StarCraft II multi-agent challenge (SMAC).

HumanLight: Incentivizing ridesharing via human-centric deep reinforcement learning in traffic signal control

Single occupancy vehicles are the most attractive transportation alternative for many commuters, leading to increased traffic congestion and air pollution. Advancements in information technologies create opportunities for smart solutions that incentivize ridesharing and mode shift to higher occupancy vehicles (HOVs) to achieve the car lighter vision of cities. In this study, we present HumanLight, a novel decentralized adaptive traffic signal control algorithm designed to optimize people throughput at intersections. Our proposed controller is founded on reinforcement learning with the reward function embedding the transportation-inspired concept of pressure at the person-level. By rewarding HOV commuters with travel time savings for their efforts to merge into a single ride, HumanLight achieves equitable allocation of green times. Apart from adopting FRAP, a state-of-the-art (SOTA) base model, HumanLight introduces the concept of active vehicles, loosely defined as vehicles in proximity to the intersection within the action interval window. The proposed algorithm showcases significant headroom and scalability in different network configurations considering multimodal vehicle splits at various scenarios of HOV adoption. Improvements in person delays and queues range from 15% to over 55% compared to vehicle-level SOTA controllers. We quantify the impact of incorporating active vehicles in the formulation of our RL model for different network structures. HumanLight also enables regulation of the aggressiveness of the HOV prioritization. The impact of parameter setting on the generated phase profile is investigated as a key component of acyclic signal controllers affecting pedestrian waiting times. HumanLight’s scalable, decentralized design can reshape the resolution of traffic management to be more human-centric and empower policies that incentivize ridesharing and public transit systems.

A large-scale traffic signal control algorithm based on multi-layer graph deep reinforcement learning

Due to its capability in handling complex urban intersection environments, deep reinforcement learning (DRL) has been widely applied in Adaptive Traffic Signal Control (ATSC). However, most existing algorithms are designed for specific road networks or traffic conditions, making it difficult to transfer them to new environments. Moreover, current graph-based algorithms do not fully capture the geometric and spatial features of intersections, leading to incomplete embedding of the agent's environment depiction. Additionally, the actions adopted by these algorithms are inherently based on fixed-cycle phases, limiting the flexibility of traffic signal control. To address the aforementioned issues, this paper proposes a Multi-layer Graph Mask Q-Learning (MGMQ) algorithm for multi-intersection ATSC to optimize traffic and reduce delay. Unlike previous graph-based algorithms, this paper introduces a method for computing multi-layer graphs, dividing the traffic environment into upper-level traffic network-layer graphs and lower-level intersection-layer graphs, and employs the graph attention algorithm and an improved GraphSAGE algorithm for computation. This method not only enables the generation of embedded state for intersections that include geometric and spatial features, but also allows the algorithm to adapt to different traffic conditions and road networks. Additionally, we introduce an action masking mechanism, allowing this algorithm can be adapted to different action spaces. As a result, the algorithm uses arbitrary signal phases as actions to achieve flexible traffic flow control, and can be directly applied to intersections with arbitrary geometry. The final test results demonstrate that a model trained solely on synthetic road networks can be directly transferred to other synthetic network configurations or real-world urban road networks, outperforming current state-of-the-art algorithms.

Fusing crowdsourced data to an adaptive wireless traffic signal control system architecture

In this study, a novel wireless adaptive traffic signal control System architecture is proposed, in which crowdsourced data is integrated to detect the traffic state on each approach of an intersection. Under the wireless architecture, the green signal timing is calculated using a travel time-based Max Pressure control algorithm, which is performed on a single-board computer (SBC, RPi 4). The resulting green timings are sent from SBC to the master controller (ESP32) over serial UART and then to the slave (ESP32) wirelessly using ESP-NOW; slave ESP32 is placed on each approach of the intersection. It integrates a fail-safe mechanism to avoid any vehicular conflicts. The proposed system architecture also embeds a crowdsourced data-based pre-emption of intersections for efficient movement of emergency vehicles. The efficacy of the proposed system is compared using a simulation tool with real-world data. With respect to actuated and Max Pressure based on traffic flow, the proposed algorithm reduces the total delay by 54.8% and 22.96%, respectively. Further, the total queue dissipation time decreases by 8.32% and 5.45%, respectively. It outperforms the actuated signal control as well as Max Pressure based on traffic flows. The capability of transforming existing traffic signal controllers into wireless adaptive traffic signal control systems makes it a very affordable and scalable solution.

π-Light: Programmatic Interpretable Reinforcement Learning for Resource-Limited Traffic Signal Control

The recent advancements in Deep Reinforcement Learning (DRL) have significantly enhanced the performance of adaptive Traffic Signal Control (TSC). However, DRL policies are typically represented by neural networks, which are over-parameterized black-box models. As a result, the learned policies often lack interpretability, and cannot be deployed directly in the real-world edge hardware due to resource constraints. In addition, the DRL methods often exhibit limited generalization performance, struggling to generalize the learned policy to other geographical regions. These factors limit the practical application of learning-based approaches. To address these issues, we suggest the use of an inherently interpretable program for representing the control policy. We present a new approach, Programmatic Interpretable reinforcement learning for traffic signal control (π-light), designed to autonomously discover non-differentiable programs. Specifically, we define a Domain Specific Language (DSL) and transformation rules for constructing programs, and utilize Monte Carlo Tree Search (MCTS) to find the optimal program in a discrete space. Extensive experiments demonstrate that our method consistently outperforms baseline approaches. Moreover, π-Light exhibits superior generalization capabilities compared to DRL, enabling training and evaluation across intersections from different cities. Finally, we analyze how the learned program policies can directly deploy on edge devices with extremely limited resources.

Stability of Traffic Equilibria in a Day-to-Day Dynamic Model of Route Choice and Adaptive Signal Control

Adaptive traffic signal control has the potential to promote the efficient use of road intersections, thus contributing to the effectiveness of urban traffic management schemes. However, the reaction of drivers to repeatedly updated signal settings and the ensuing route choice dynamics may trigger the emergence of various kinds of network instability. In this study, the joint evolution of traffic flows and adaptive signal settings in a road network is investigated at the level of day-to-day dynamics with an explicit focus on the stability issue. We show how a Logit form signal control policy can be used, in interaction with route choice, to counter the emergence of instabilities possibly arising as a consequence of various behavioral factors and network conditions. After providing a general formulation of the model as a discrete time, deterministic nonlinear dynamical system, an explicit analysis of fixed-point stability is carried out for a simple network. Numerical results obtained from the implementation of the model on two example networks are presented in order to support the analytical findings of this study. We conclude that, in an integrated traffic management and information system, a properly calibrated adaptive signal control policy has the potential to offset the destabilizing effect of highly accurate driver information supplied by navigational aids. Our findings also suggest that the Logit-like control policy performs better than the Equisaturation signal setting method, in terms of average intersection delay at equilibrium, for all levels of driver information and travel demand tested in the experiment.

Advanced Drone Surveillance System for Traffic Management

The rapid urbanization and increasing vehicle traffic in metropolitan areas have given rise to a significant challenges in traffic management. Traditional methods of a traffic monitoring and control often struggle to cope with the dynamic and complex nature of urban traffic patterns. In this context, we propose an innovative Drone Surveillance System for Traffic Management to revolutionize the way traffic is monitored, analyzed, and optimized in urban environments. Our system leverages state-of-the-art drone technology equipped with advanced sensors, high-resolution cameras, and intelligent algorithms to provide real-time traffic insights. Drones are deployed strategically across key traffic arteries, allowing for comprehensive aerial coverage and data collection. The system focuses on several key objectives: real-time traffic monitoring, incident detection, traffic flow analysis, optimized traffic signal control, emergency response support, data-driven decision making, and ensuring privacy and security compliance. By the employing cutting-edge computer vision algorithms, machine learning techniques, and data analytics, the Drone Surveillance System processes vast amounts of traffic data with speed and precision. The system detects the traffic incidents and the such as accidents and road hazards promptly, enabling swift a responses from emergency services. It analyzes traffic patterns and predicts congestion, allowing for adaptive traffic signal control and dynamic route optimization, thereby minimizing congestion and reducing travel time for a commuters. One of the system’s notable features is its a ability to the support emergency response teams effectively. Drones identify incidents and relay on the critical information to emergency services, International Journal of Scientific Research in Engineering and Management (IJSREM) Volume: 08 Issue: 02 | February - 2024 SJIF Rating: 8.176 ISSN: 2582-3930 © 2024, IJSREM | www.ijsrem.com DOI: 10.55041/IJSREM28534 | Page 2 enabling rapid deployment and efficient management of emergencies. Moreover, the system ensures the privacy of individuals through strict adherence to regulations and deploys robust security measures to safeguard data integrity. An important advancement in the urban traffic control system is the Drone Surveillance System for Traffic control. The technology turns traditional traffic management into a proactive, flexible, and data-driven procedure by utilizing the power of drones, real-time data processing, and sophisticated algorithms. This innovation not only makes urban transportation more efficient, but it also helps to improve road safety by lowering accident rates, easing traffic, and creating a more sustainable urban environment. Keywords: Traffic Management, Real-time Monitoring, Incident Detection, Traffic Flow Analysis, Optimized Traffic Signal Control, Machine Learning Techniques. Drone Surveillance System

Elevating adaptive traffic signal control in semi‐autonomous traffic dynamics by using connected and automated vehicles as probes

AbstractIn this work, the connected vehicle's messages are used to create an enhanced adaptive traffic signal control (ATSC) system for improved traffic flow. Few existing studies use connected and automated vehicles (CAVs) to develop traffic signal control algorithms under hybrid connected and autonomous conditions. The proposed approach focuses on a four‐phase traffic intersection with both CAVs and human‐driven vehicles (HVs). CAVs share real‐time state information, and a model called Roads Dynamic Segmentation estimates queuing procedures and vehicle fleet numbers on dynamic road sections. This information is used in the Store and Forward Model (SFM) to predict intersection queuing length. The ATSC system, based on model predictive control (MPC), aims to minimize intersection queue length while considering traffic constraints (undersaturated, saturated, and oversaturated) and avoiding free‐flow problems due to queue overflow. To reduce computational complexity, a linear‐quadratic‐regulator (LQR) is used. Real‐world vehicle trajectories and the SUMO tool are used for experimental purposes. Results show that the proposed method reduces average delay by 38.50% and 33.42% compared to fixed timing and traditional MPC in cases of oversaturated traffic flow with 100% CAV penetration. Even with a penetration rate of only 20%, average delay decreases by 13.65% and 6.50%, respectively. This study showcases not only the potential benefits of CAV in enhancing traffic, but also enables the optimal utilization of green duration in signalized intersection control systems. This helps prevent traffic congestion and ensures the efficient and smooth movement of traffic flow.

Enhancing Robustness of Deep Reinforcement Learning Based Adaptive Traffic Signal Controllers in Mixed Traffic Environments Through Data Fusion and Multi-Discrete Actions

Enhancing Robustness of Deep Reinforcement Learning Based Adaptive Traffic Signal Controllers in Mixed Traffic Environments Through Data Fusion and Multi-Discrete Actions

Identification of optimal locations of adaptive traffic signal control using heuristic methods

Adaptive Traffic Signal Control (ATSC) adjusts signal timings to real-time traffic measurements, increasing operational efficiency within a network. However, ATSC is both expensive to install and operate making it infeasible to deploy at all signalized intersections within a network. This study presents a bi-level optimization framework that applies heuristic methods to identify a limited set of locations for ATSC deployment within an urban network. At the upper-level, the Population Based Incremental Learning (PBIL) algorithm is employed to generate, evaluate, learn, and update different ATSC configurations. The lower-level uses the delay-based Max-Pressure algorithm to simulate the ATSC configuration within a microsimulation platform. The study proposes improvements to the PBIL algorithm by considering constraints on the maximum number of intersections for ATSC deployment and incorporates prior information about the intersection performance (i.e., informed search). Simulation results on the traffic network of State College, PA reveal that the proposed PBIL algorithm consistently outperforms baseline methods that select locations only based on queue-lengths or delays in terms of reducing overall network travel times. The study also reveals that intersections experiencing the highest delays or longest queues are not always the best candidates for ATSC. Moreover, applying ATSC at all intersections does not always provide the best performance; in fact, ATSC applied to some locations could increase travel times by contributing additional congestion downstream. Additionally, the modified PBIL algorithm with the informed search strategy is more efficient at identifying promising solutions suggesting it can be readily applied to more generalized optimization problems.

Privacy-preserving adaptive traffic signal control in a connected vehicle environment

Although Connected Vehicles (CVs) have demonstrated tremendous potential to enhance traffic operations, they can impose privacy risks on individual travelers, e.g., leaking sensitive information about their frequently visited places, routing behavior, etc. Despite the large body of literature that devises various algorithms to exploit CV information, research on privacy-preserving traffic control is still in its infancy. In this paper, we aim to fill this research gap and propose a privacy-preserving adaptive traffic signal control method using partially connected vehicle data. Specifically, we proposed a privacy-preserving mechanism to protect CV data against three types of attacks: CV collusion attacks, database attacks, and inference attacks. The mechanism leverages secure Multi-Party Computation and differential privacy to aggregate individual-level CV data to calculate key traffic parameters without compromising the privacy of CV users. For seamless integration with the privacy-preserving mechanism, we develop a traffic signal optimization model and an arrival rate estimator relying only on aggregated CV data, being applied to both undersaturated and oversaturated traffic conditions. The optimization model is further extended to a stochastic programming problem to explicitly handle the noises added by the privacy-preserving mechanism. Evaluation results show that the linear optimization model preserves privacy with a marginal impact on control performance, and the stochastic programming model can significantly reduce residual queues compared to the linear programming model, with almost no increase in vehicle delay. Overall, our methods demonstrate the feasibility of incorporating privacy-preserving mechanisms in CV-based traffic modeling and control, which guarantees both utility and privacy.

Mean Field Multi-Agent Reinforcement Learning Method for Area Traffic Signal Control

Reinforcement learning is an effective method for adaptive traffic signal control in urban transportation networks. As the number of training rounds increases, the optimal control strategy is learned, and the learning capabilities of deep neural networks are further enhanced, thereby avoiding the limitations of traditional signal control methods. However, when faced with the sequential decision tasks of regional signal control, it encounters issues such as the curse of dimensionality and environmental non-stationarity. To address the limitations of traditional reinforcement learning algorithms applied to multiple intersections, the mean field theory is applied. This models the traffic signal control problem at multiple intersections within a region as interactions between individual intersections and the average effects of neighboring intersections. By decomposing the Q-function through bilateral estimation between the agent and its neighbors, this method reduces the complexity of interactions between agents while preserving global interactions between the agents. A traffic signal control model based on Mean Field Multi-Agent Reinforcement Learning (MFMARL) was constructed, containing two algorithms: Mean Field Q-Network Area Traffic Signal Control (MFQ-ATSC) and Mean Field Actor-Critic Network Area Traffic Signal Control (MFAC-ATSC). The model was validated using the SUMO simulation platform. The experimental results indicate that across different metrics, such as average speed, the mean field reinforcement learning method outperforms classical signal control methods and several existing approaches.

Multi-Objective Traffic Signal Control Using Network-Wide Agent Coordinated Reinforcement Learning

The optimization of intersection signal control can improve traffic efficiency, reduce congestion degree, and improve traffic safety. Aiming at implementing the coordinated adaptive traffic signal control (ATSC) across large-scale arterial network, multi-agent reinforcement learning (MARL) has been widely concerned and lucubrated. Nevertheless, the existing MARL-based ATSC studies suffers from several limitations: (1) While most existing researches focused on the mobility performance of controlled corridor, there calls for a methodology that aims at combine multi-objective performance on traffic safety, efficiency, and network coordination simultaneously; (2) Most methods ignore the correlations between multiple agents, nor considers the spatial-temporal dependencies among the corelated neighboring intersections due to high communications requirements, which can hardly be achieved in real adaptive coordination control. To overcome the aforementioned difficulties, a multi-objective reinforcement learning model (NACRL) for network-wide coordinated signal control is proposed. Firstly, to enforce a coordinated network control with safety and efficiency considerations, a reward mechanism inspecting both traffic safety and traffic efficiency indicators was designed to achieve ideal performance in terms of mobility, safety and smooth. Secondly, the proposed NACRL conducted a centralized training-decentralized execution framework, this overcomes the critical limitation of data transmission in the field implementation while explicitly analyzing the traffic state over the entire network instead of examining each isolated intersection. Last but not least, the proposed algorithm utilized the attention mechanism to dynamically capture the sophisticated spatial-temporal dependencies over the complex arterial network, which aids the better coordinated control over multi-agents deployed at the intersections across the corridor. To testify the effeteness of the proposed algorithm, extensive experiments were implemented in both large-scale synthetic traffic grid and real-world arterial network. The experiment demonstrated that the proposed NACRL algorithm outperforms other state-of-the-art baselines with simultaneously improved performance in terms of traffic safety, traffic efficiency and network coordination, as well as improved algorithm convergence and interpretability.

Managing mixed traffic at signalized intersections: An adaptive signal control and CAV coordination system based on deep reinforcement learning

Managing the mixed traffic involving connected and autonomous vehicles (CAVs) and human-driven vehicles (HVs) at a signalized intersection has become a concern of researchers. However, the performances of most existing control methods are limited, especially when CAV penetration rate is low, since they fail to make a better trade-off between safety and operational efficiency for both CAVs and HVs. To this end, this study proposes a deep reinforcement learning (DRL) powered control system for the mixed traffic at signalized intersections, which aims to optimize operational efficiency of both CAVs and HVs while assuring safety and reducing interference on HVs’ driving habits. The system adopts an adaptive traffic signal control strategy and an efficient CAV control policy with a passing rule proposed as a link in between. The traffic signal control strategy allows traffic light to adaptively adjust its phase and duration based on real-time traffic information, while the CAV control policy permits the CAVs meeting certain safety constraints to form platoons and pass through the intersection in a coordinated manner regardless of traffic signals. As an efficient DRL algorithm, Deep Q-Network (DQN) is adopted to adaptively control the signals and implement CAV coordination. The proposed system is examined on Simulation of Urban Mobility (SUMO), given different CAV penetration rates and traffic conditions. It is found that the proposed system not only outperforms the state-of-the-art control methods on reducing travel time and fuel consumption under low CAV penetration rate, but also enlarges its advantages with the increase of CAV penetration rate. In certain traffic scenarios, the proposed system can even achieve a maximum reduction of travel time by 37.33% and fuel consumption by 15.95%, in comparison to the existing method with the best performance. Besides, to some extent, the comparisons between the performances of CAVs and HVs demonstrate certain benefits of introducing CAVs into the mixed traffic.

CAV-enabled data analytics for enhancing adaptive signal control safety environment

Given the connected and autonomous vehicle (CAV) generated trajectories as a “floating sensor” data source to obtain high resolution CAV-generated mobility data at intersections, to ensure maximum safety effect while maintaining efficient operations at the same time is actually a complex task in traffic management. Literature indicates that methods for evaluating the CAV-generated data potentials focusing on safety benefits are still immature. The primary reason lies in lack of underlying mechanism and data models to make the data intelligent to enhance safety environment through adaptive traffic signal control. On top of the developed intelligent CAV-generated mobility data fusion model framework in support of adaptive traffic signal control, parameters and models included in Surrogate Safety Assessment Model (SSAM) are integrated to indicate the risk of near crashes and then evaluate the safety environment. A proof-of-concept study is conducted in Uptown Cincinnati, Ohio to test the developed data fusion models in terms of safety enhancement, along with operational benefits. In the tests, the CAV-generated data supported developed adaptive signal plan is compared with the basic signal plans (i.e., pretimed signal plan, actuated signal plan) that supported by traditional detection systems. The results indicate that the adaptive signal plan has a great potential to decrease at most 91% of total collision risk (measured in probability), 71% of crossing collision risk, 90% of rear end collisions risk and 100% of lane-changing collisions risk, compared with basic signal plans. Meanwhile, it increases up to 6.8% of throughput, and decreases up to 91.49% of average delay, 96.23% of queue length and 75.00% of number of stops. The benefits of operation efficiency include reduced average delay and reduced number of stops; but no improvement in reducing collisions severity that is reflected by high maximum speed and relative speed of two vehicles involved in a potential collision.

Cycle-based signal timing with traffic flow prediction for dynamic environment

This article studies adaptive traffic signal control problem of single intersection in dynamic environment. A novel cycle-based signal timing method with traffic flow prediction (CycleRL) is proposed to improve the traffic efficiency under dynamic traffic flow. Firstly, the empirical mode decomposition is applied to denoise the flow data. Then a data-model hybrid driven traffic flow prediction strategy is designed to predict the traffic flow, which combines a model-based Kalman filter and an LSTM network-based predictor and adopts another Kalman filter to fuse both prediction results to improve the prediction precision. Besides, a robust signal cycle timing strategy based on human-machine collaboration is developed to deal with dynamic traffic flow, which firstly designs a rule-based signal cycle scheme according to the predicted flow data as the preliminary scheme, and then finetunes the preliminary scheme based on Soft Actor-Critic (SAC) algorithm according to the real-time traffic dynamics. The experiments in both synthetic scenario and real-world scenario show that the proposed data-model hybrid driven traffic flow prediction algorithm has better prediction performance and the proposed CycleRL method outperforms rule-based methods, flow-based allocation methods and traditional reinforcement learning method. Moreover, it is also shown that the proposed CycleRL method has better transferability to bridge the discrepancy across domains.