Traffic Light Control Algorithm Research Articles

A Survey on Challenges and Solutions of Traffic Surveillance Systems in IoT-Enabled Eco System

A Survey on Challenges and Solutions of Traffic Surveillance Systems in IoT-Enabled Eco System

Traffic light control with reinforcement learning

Urban traffic signal optimization is important for alleviating congestion in urban transportation systems. This study proposes a real-time traffic light control algorithm based on deep Q learning with a reward function that accounts for queue lengths, delays, travel times, and throughput. The model dynamically decides phase changes based on current traffic conditions. The training of the deep Q network involves an offline stage from pre-generated data with fixed signal timing and an online stage using real-time traffic data. A deep Q network structure with aphase gate component is used to simplify the model's learning task under different phases. Amemory palace" mechanism is used to address sample imbalance during the training process. Both synthetic and real-world traffic flow data are used to validate our approach under an urban road intersection scenario in Hangzhou, China. Results demonstrate significant performance improvements of the proposed method in reducing vehicle waiting time (57.1% to 100%), queue lengths (40.9% to 100%), and total travel time (16.8% to 68.0%) compared to traditional fixed signal timing plans.

DERLight: A Deep Reinforcement Learning Traffic Light Control Algorithm with Dual Experience Replay

<p>In recent years, with the increasingly severe traffic environment, most cities are facing various traffic congestion problems, and the demand for intelligent regulation of traffic signals is also increasing. In this study, we propose a new intelligent traffic light control algorithm, dual experience replay light (DERLight), which innovatively and efficiently designs a dual experience replay training mechanism based on the classic deep Q network (DQN) framework and considers the dynamic epoch function. As results show that compared with some state-of-the-art algorithms, DERLight can shorten the average travel time of vehicles, increase the throughput at intersections, and also speed up the convergence of the network. In addition, the design of this algorithm framework is not only limited to the field of intelligent transportation, but also has transferability for some other fields.</p> <p> </p>

A traffic light control method based on multi-agent deep reinforcement learning algorithm

Intelligent traffic light control (ITLC) algorithms are very efficient for relieving traffic congestion. Recently, many decentralized multi-agent traffic light control algorithms are proposed. These researches mainly focus on improving reinforcement learning method and coordination method. But, as all the agents need to communicate while coordinating with each other, the communication details should be improved as well. To guarantee communication effectiveness, two aspect should be considered. Firstly, a traffic condition description method need to be designed. By using this method, traffic condition can be described simply and clearly. Secondly, synchronization should be considered. As different intersections have different cycle lengths and message sending event happens at the end of each traffic signal cycle, every agent will receive messages of other agents at different time. So it is hard for an agent to decide which message is the latest one and the most valuable. Apart from communication details, reinforcement learning algorithm used for traffic signal timing should also be improved. In the traditional reinforcement learning based ITLC algorithms, either queue length of congested cars or waiting time of these cars is considered while calculating reward value. But, both of them are very important. So a new reward calculation method is needed. To solve all these problems, in this paper, a new ITLC algorithm is proposed. To improve communication efficiency, this algorithm adopts a new message sending and processing method. Besides, to measure traffic congestion in a more reasonable way, a new reward calculation method is proposed and used. This method takes both waiting time and queue length into consideration.

Multi-agent deep reinforcement learning with actor-attention-critic for traffic light control

In recent years, with the increase of urbanization and car ownership, urban traffic congestion have become increasingly prominent. Traffic light control can effectively reduce urban traffic congestion. In the research of controlling traffic lights of multiple intersections, most methods introduced theories related to deep reinforcement learning, but few methods considered the information interaction between intersections or the way of information interaction is unreasonable. Inspired by this, this paper proposes a multi-agent deep reinforcement learning with actor-attention-critic network for traffic light control (MAAC-TLC) algorithm. In MAAC-TLC, each agent introduces the attention mechanism in the process of learning, so that it will not pay attention to all the information of other agents indiscriminately, but only focus on the important information of the agents that plays an important role in it, so as to ensure that all intersections can learn the optimal policy. Finally, the traffic lights at each intersection in the MAAC-TLC algorithm are controlled according to its own policy, thereby improving the traffic efficiency of the traffic network. The experimental results on light and heavy traffic flow scenarios have demonstrated that MAAC-TLC can improve traffic congestion at multiple intersections effectively.

Analytically Guided Reinforcement Learning for Green It and Fluent Traffic

This study investigates various methods for autonomous traffic signal control. We look into different types of control methods, including fixed time, adaptive, analytic, and reinforcement learning approaches. Machine learning approaches are compared with the “analytic” approach, which is used as “gold standard” for performance assessment. We find that conventional machine learning approaches are better than the analytic approach, but require a lot more computer power. We, therefore, introduce a novel hybrid method called “analytically guided reinforcement learning” or shorter “α-RL”. This approach is implemented in our “GuidedLight agent” and tends to outperform both, classical machine learning and the analytic approach, while largely improving convergence. This method is therefore suited as a “green IT” solution that improves environmental impact in a two-fold way: by reducing (i) traffic congestion and (ii) the processing power needed for the learning and operation of the traffic light control algorithm.

Fog-based dynamic traffic light control system for improving public transport

Nowadays, traffic congestion is increasingly aggravating in almost every urban area and existing traffic light controllers cannot satisfy the rising demands efficiently to handle the traffic pressure. An intelligent transportation system (ITS) aims to provide major innovations for improving the performance of a public transport system including the traffic light control systems. In this paper, we propose a fog-based distributed architecture for a dynamic traffic light control system for ITS. In the proposed architecture, the local gateway collects real-time local traffic data using a wireless sensor network at each intersection. It also gets the neighboring traffic data from the distributed fog nodes. Using this local and global traffic data, the proposed Efficient Dynamic Traffic Light Control algorithm for Multiple (EDTLCM) intersections calculates the optimal green light sequence and duration (that maximizes the benefits), considering three objectives i) reducing the average waiting time ii) minimizing the fuel consumption and iii) maximizing the throughput. The simulation result shows that the proposed EDTLCM minimizes the waiting time, reduces fuel consumption and improves system throughput compared to the other dynamic strategies.

Situation-cognitive traffic light control based on object detection using YOLO algorithm

Current traffic lights provide the green signal with fixed-time interval without considering the traffic situation. As a result, cars in a long line have to wait long time, which causes traffic jams and makes the drivers be irritated. In order to solve the problem, it is necessary to control the green signal interval according to the analysed traffic volume using the image processing and the machine learning techniques. This paper presents a situation-cognitive traffic light control algorithm that measures the traffic volume using object detection algorithm called you only look once (YOLO) and controls the traffic signal intervals according to the traffic volume. The algorithm expects the smooth traffic flow and the reduction of the driver's stress.

Situation-cognitive traffic light control based on object detection using YOLO algorithm

Current traffic lights provide the green signal with fixed-time interval without considering the traffic situation. As a result, cars in a long line have to wait long time, which causes traffic jams and makes the drivers be irritated. In order to solve the problem, it is necessary to control the green signal interval according to the analysed traffic volume using the image processing and the machine learning techniques. This paper presents a situation-cognitive traffic light control algorithm that measures the traffic volume using object detection algorithm called you only look once (YOLO) and controls the traffic signal intervals according to the traffic volume. The algorithm expects the smooth traffic flow and the reduction of the driver's stress.

Distributed Cooperative Backpressure-Based Traffic Light Control Method

On the foundation of the original backpressure-based traffic light control algorithm, a distributed cooperative backpressure-based traffic light control method is proposed in this paper. The urban traffic network is modeled as a smart agent-controlled queuing network, in which the intersection agents exchange the queue length information and the selected activating light phase information of neighboring intersections through communications and determine the activating light phase at each time slot according to local traffic information. The improved phase pressure computation method considers the phase state of downstream intersections instead of only the queue length of the local intersections. Light phase switching coordination among adjacent intersections is achieved using the consensus-based bundle algorithm, in which the cooperative light phase switching problem is viewed as a task assignment issue among adjacent intersections. Simulation results illustrated that the proposed cooperative backpressure-based traffic light control method obtained better performance than the original backpressure-based and fixed-time traffic control methods.

Autonomous and connected intersection crossing traffic management using discrete-time occupancies trajectory

In this paper, we address the problem of safe and efficient intersection crossing traffic management of autonomous and connected ground traffic. Toward this objective, we propose an algorithm called the discrete-time occupancies trajectory based intersection traffic coordination algorithm (DICA). We show that the basic DICA has a computational complexity of $\mathcal {O}(n^{2} {L_{m}^{3}})$ where n is the number of vehicles granted to cross an intersection and Lm is the maximum length of intersection crossing routes. To improve the overall computational efficiency of the algorithm, the basic DICA is enhanced by several computational approaches that are proposed in this paper. The enhanced algorithm has the computational complexity of $\mathcal {O}(n^{2} L_{m} \log _{2} L_{m})$ . The improved computational efficiency of the enhanced algorithm is validated through simulations using an open source traffic simulator called the simulation of urban mobility (SUMO). The overall throughput, as well as the computational efficiency of the enhanced algorithm, are also compared with those of an optimized traffic light control algorithm.

Using an Agent‐Based Simulation to Evaluate the Performance of Control Algorithms for an Intelligent Traffic Light System

AbstractThis paper examines a case study for using an agent‐based model to evaluate potential solutions to the ubiquitous problem of automobile traffic. The overall objective is to use simulations to evaluate the performance of intelligent traffic light system control algorithms to improve traffic flow along an entire section of highway. The approach uses an agent‐based modeling tool to simulate the flow of traffic using several different traffic control algorithms. The methodology includes steps to define the measures of effectiveness, simulation parameters, traffic control algorithms, and identify a simulation tool, followed by simulating the algorithms, and evaluating the performance results. Several alternative algorithms were evaluated including single‐intersection control and multi‐intersection synchronized flow. The initial results show a significant performance difference between the control algorithms. However, simulated traffic conditions need to be validated, along with further model refinements, before a final assessment of the algorithm performance can be made.

Advantage of cooperative traffic light control algorithms

Based on the presentation at ITS Europe Congress Glasgow 2016. Contemporary traffic light control (TLC) systems rely on sensors for detection of traffic which are costly in purchase, installation and maintenance. Emerging cooperative technology offers an attractive alternative where only one road side unit per intersection is required, instead of several infrastructure sensors per lane. However, studies showed that traffic control with cooperative detection requires a penetration rate of at least 20% to function effectively. To show the potential of cooperative traffic control, this study presents three algorithms: (i) the SWARM control algorithm, which is designed to work with very low penetration rates; (ii) an extension to the adaptive control algorithm, ImFlow, which uses cooperative data for enhanced queue modelling; and (iii) an ImFlow extension to stabilise green planning to enable green light optimal speed advice. The results from micro-simulation show a 7.8% improvement for stops and delay time over traditional adaptive control for SWARM, and 14.9% for Cooperative ImFlow. Adding planning stabilisation reduced the average perceived change for end users from 9.0-2.3%, without performance loss for the overall traffic flow. This shows the large potential of cooperative traffic control.

Earliest-Deadline-Based Scheduling to Reduce Urban Traffic Congestion

One of the major problems, caused by traffic congestion, owes its existence to the unwanted delay experienced by the priority vehicles. The evaluation of two scheduling algorithms as adaptive traffic control algorithms has been proposed here to reduce this unwanted delay. One of these algorithms is the earliest deadline first (EDF) algorithm, whereas the other is the fixed priority (FP) algorithm. The performance of both algorithms as adaptive traffic light control algorithms is evaluated for isolated traffic intersections. A comparative study is performed here, where the performance of these algorithms is compared against a fixed static traffic light controller. Moreover, their performance is also compared against each other. Conclusive results from the simulation of the algorithms reveal that the number of stops, average delay, and mean trip time of the priority vehicles is significantly reduced by the implementation of these algorithms. Furthermore, it has been shown that the overall performance of EDF is much better than FP in terms of improvement of different performance measures for congestion reduction of priority vehicles.

Threshold Tuning Using Stochastic Optimization for Graded Signal Control

Adaptive control of traffic lights is a key component of any intelligent transportation system. Many real-time traffic light control (TLC) algorithms are based on graded thresholds, because precise information about the traffic congestion in the road network is hard to obtain in practice. For example, using thresholds <formula formulatype="inline" xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex Notation="TeX">$L_{1}$</tex></formula> and <formula formulatype="inline" xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex Notation="TeX">$L_{2}$</tex> </formula> , we could mark the congestion level on a particular lane as “low,” “medium,” or “high” based on whether the queue length on the lane is below <formula formulatype="inline" xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex Notation="TeX">$L_{1}$ </tex></formula> , between <formula formulatype="inline" xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex Notation="TeX">$L_{1}$</tex></formula> and <formula formulatype="inline" xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex Notation="TeX">$L_{2}$</tex></formula> , or above <formula formulatype="inline" xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex Notation="TeX">$L_{2}$</tex> </formula> , respectively. However, the TLC algorithms that were proposed in the literature incorporate fixed values for the thresholds, which, in general, are not optimal for all traffic conditions. In this paper, we present an algorithm based on stochastic optimization to tune the thresholds that are associated with a TLC algorithm for optimal performance. We also propose the following three novel TLC algorithms: 1) a full-state Q-learning algorithm with state aggregation, 2) a Q-learning algorithm with function approximation that involves an enhanced feature selection scheme, and 3) a priority-based TLC scheme. All these algorithms are threshold based. Next, we combine the threshold-tuning algorithm with the three aforementioned algorithms. Such a combination results in several interesting consequences. For example, in the case of Q-learning with full-state representation, our threshold-tuning algorithm suggests an optimal way of clustering states to reduce the cardinality of the state space, and in the case of the Q-learning algorithm with function approximation, our (threshold-tuning) algorithm provides a novel feature adaptation scheme to obtain an “optimal” selection of features. Our tuning algorithm is an incremental-update online scheme with proven convergence to the optimal values of thresholds. Moreover, the additional computational effort that is required because of the integration of the tuning scheme in any of the graded-threshold-based TLC algorithms is minimal. Simulation results show a significant gain in performance when our threshold-tuning algorithm is used in conjunction with various TLC algorithms compared to the original TLC algorithms without tuning and with fixed thresholds.

An Open Traffic Light Control Model for Reducing Vehicles' $\hbox{CO}_{2}$ Emissions Based on ETC Vehicles

Usually, vehicles' stop-and-go driving will consume more fuel and emit more CO <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">2</sub> than constant speed driving. To reduce vehicles' CO <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">2</sub> emissions, vehicles' travel should be smoothed by reducing the stop-and-go times. In this paper, a three-tier structure is proposed to realize dynamic traffic light control for smoothing vehicles' travel. In tier-1, an electronic toll collection (ETC) system is employed for collecting road traffic flow data and calculating the recommended speed. In tier-2, radio antennas are installed near the traffic lights. Road traffic flow information can be obtained by wireless communication between the antennas and ETC devices. In tier-3, a branch-and-bound-based real-time traffic light control algorithm is designed to smooth vehicles' travels. After smoothing vehicles' travels, more vehicles can pass intersections with less waiting time and fewer short-time stops; therefore, the vehicles' CO <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">2</sub> emissions can be reduced. Simulation results indicate that the proposed scheme performs much better than the adaptive fuzzy traffic light control method: The average waiting time, short-time stop times, and CO <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">2</sub> emissions are greatly reduced, and the nonstop passing rate is greatly improved.