Adaptive Traffic Signal Control Research Articles

Efficiency of Vehicle Actuated traffic control on isolated intersection

This paper is described different strategies for traffic control and simulation, namely - Vehicle Actuated (VA) compared to Fixed Time (FT) traffic control. We are comparing and evaluating the impact of FT and adaptive traffic signal control strategies for one of the biggest and busiest intersections in Tashkent, the capital of Uzbekistan. The parameters used for comparing these strategies are the average vehicle travel times and average vehicle queue lengths. TRL TRANSYT 15 software was used to determine the fixed signal timing plan based on intersection geometry, inter-green matrix, and traffic flow. PTV VISSIM software is used for VA algorithm development and simulation to get the maximum and average vehicle travel times as well as queue lengths. It is shown that the parameter optimization can reduce the queue lengths down to 65% and the travel time down to 51%.

A citywide TD‐learning based intelligent traffic signal control for autonomous vehicles: Performance evaluation using SUMO

AbstractAn autonomous vehicle can sense its environment and operate without human involvement. Its adequate management in an intelligent transportation system could significantly reduce traffic congestion and overall travel time in a network. Adaptive traffic signal controller (ATSC) based on multi‐agent systems using state‐action‐reward‐state‐action (SARSA ()) are well‐known state‐of‐the‐art models to manage autonomous vehicles within urban areas. However, this study found inefficient weights updating mechanisms of the conventional SARSA () models. Therefore, it proposes a Gaussian function to regulate the eligibility trace vector's decay mechanism effectively. On the other hand, an efficient understanding of the state of the traffic environment is crucial for an agent to take optimal actions. The conventional models feed the state values to the agents through the MinMax normalization technique, which sometimes shows less efficiency and robustness. So, this study suggests the MaxAbs scaled state values instead of MinMax to address the problem. Furthermore, the combination of the A‐star routing algorithm and proposed model demonstrated a good increase in performance relatively to the conventional SARSA ()‐based routing algorithms. The proposed model and the baselines were implemented in a microscopic traffic simulation environment using the SUMO package over a complex real‐world‐like ‐intersections network to evaluate their performance. The results showed a reduction of the vehicle's average total waiting time and total stops by a mean value of % and % compared to the considered baselines. Also, the A‐star combined with the proposed controller outperformed the conventional approaches by increasing the vehicle's average trip speed by %.

Cooperative Control for Signalized Intersections in Intelligent Connected Vehicle Environments

Cooperative control of vehicle trajectories and traffic signal phases is a promising approach to improving the efficiency and safety of transportation systems. This type of traffic flow control refers to the coordination and optimization of vehicle trajectories and traffic signal phases to reduce congestion, travel time, and fuel consumption. In this paper, we propose a cooperative control method that combines a model predictive control algorithm for adaptive traffic signal control and a trajectory construction algorithm. For traffic signal phase selection, the proposed modification of the adaptive traffic signal control algorithm combines the travel time obtained using either the vehicle trajectory or a deep neural network model and stop delays. The vehicle trajectory construction algorithm takes into account the predicted traffic signal phase to achieve cooperative control. To evaluate the method performance, numerical experiments have been conducted for three real-world scenarios in the SUMO simulation package. The experimental results show that the proposed cooperative control method can reduce the average fuel consumption by 1% to 4.2%, the average travel time by 1% to 5.3%, and the average stop delays to 27% for different simulation scenarios compared to the baseline methods.

Cooperative Control of Traffic Signals and Vehicle Trajectories

The transportation system is one of the most important parts of the country's economy. At the same time, the growth in road traffic has a significant negative impact on the economic performance of the industry. One of the ways to increase the efficiency of using the transportation infrastructure is to manage traffic flows, incl. by controlling traffic signals at signalized intersections. One of the trends in the development of intelligent transportation systems is the creation of vehicular ad hoc networks that allow the exchange of information between vehicles and infrastructure, as well as the development of autonomous vehicles. As a result, it becomes possible to formulate the problem of cooperative control of vehicle trajectories and traffic signals to increase the capacity of intersections and reduce fuel consumption and travel time. This paper presents a method for managing traffic flow at an intersection, which consists of the cooperative control of traffic signals and trajectories of connected/autonomous vehicles. The developed method combines an algorithm for the adaptive control of traffic signals based on a deterministic model for predicting the movement of vehicles and a two-stage algorithm for constructing the trajectory of vehicles. The objective optimization function used to construct the optimal trajectories takes into account fuel consumption, travel time on the road lane, and waiting time at the intersection. Experimental studies of the developed method were carried out in the microscopic traffic simulation package SUMO using three simulation scenarios, including two synthetic scenarios and a scenario in a real urban environment. The results of experimental studies confirm the effectiveness of the developed method in terms of fuel consumption, travel time, and waiting time in comparison with the adaptive traffic signal control algorithm.

Adaptive traffic signal control using deep Q-learning: case study on optimal implementations

Deep reinforcement learning has found great successes in addressing many challenging control problems; however, real-world implementations are still scarce if not non-existent. This is primarily due to three main challenges pertaining to the implementation, the stability, the optimal settings, and a lack of knowledge on methods that can be applied to field settings. This research attempts to address these issues with an adaptive simulation-based control framework proposed specifically for the training and evaluation. The control framework has implemented simulation models to conduct an extensive sensitivity analysis on the effects of key design variables, including rewarding schemes, state spaces, and model training parameters. The feasibility of transfer learning as a training strategy is also studied on scenarios with different layouts and different driver behavior models. Complex scenarios are also evaluated and used as test cases, including multiphase ring-and-barrier control and multi-intersection control. The research has contributed a significant amount of evidence on several critical design and implementation-related questions such as input representation, data (technology) requirements, training methods, and model transferability.

A Resilient Intelligent Traffic Signal Control Scheme for Accident Scenario at Intersections via Deep Reinforcement Learning

Deep reinforcement learning methods have shown promising results in the development of adaptive traffic signal controllers. Accidents, weather conditions, or special events all have the potential to abruptly alter the traffic flow in real life. The traffic light must take immediate and appropriate action based on a reasonable understanding of the environment. In this way, traffic congestion would be prevented. In this paper, we develop a reliable controller for such a highly dynamic environment and investigate the resilience of these controllers to a variety of environmental disruptions, such as accidents. In this method, the agent is provided with a complete understanding of the environment by discretizing the intersection and modifying the state space. The proposed algorithm is independent of the location and time of accidents. If the location of the accident changes, the agent does not need to be retrained. The agent is trained using deep Q-learning and experience replay. The model is evaluated in the traffic microsimulator SUMO. The simulation results demonstrate that the proposed method is effective at shortening queues when there is disruption.

Traffic signal priority control based on shared experience multi‐agent deep reinforcement learning

AbstractDeep Reinforcement Learning (DRL) has demonstrated its great potential for Adaptive Traffic Signal Control (ATSC) tasks at single‐intersection. In the transportation network multi‐agent environment, cooperative learning among multi‐agents has become a hot research topic. Based on the distributed control model, this paper presents a hybrid reward function model for the dynamic density method of intersections, which emphasizes the priority of emergency vehicles (EMV) while maximizing the traffic efficiency of social vehicles, and solves the problem of sparse reward due to the ambiguous guidance relationship between the multi‐agent Deep Reinforcement Learning (MDRL) state and reward function of the urban road network scenario. On the other hand, based on multi‐agent A2C (MA2C) algorithm, this paper presents Shared Experience MA2C (SEMA2C) between agents. In the transportation network, each intersection represented by an agent has similar task objectives. SMEA2C algorithm takes the current agent as the main body of self‐learning, and utilizes the principle of importance sampling to learn from the experience data of the agents located at adjacent intersections. The experimental results show that the proposed SEMA2C performs well in multi‐agent traffic signal control tasks, and has greater advantages than the similar algorithms.

Neighborhood Cooperative Multiagent Reinforcement Learning for Adaptive Traffic Signal Control in Epidemic Regions

Nowadays, multiagent reinforcement learning (MARL) have shared significant advances in the adaptive traffic signal control (ATSC) problems. For most of the researches, agents are all isomorphic, which disregards the situation in which isomerous intersections cooperative together in a real ATSC scenario, especially in epidemic regions where different intersections have quite different levels of importance. To this end, this paper models the ATSC problem as a networked Markov game (NMG), in which agents take into account information, including traffic conditions of it and its connected neighbors. A cooperative MARL framework named neighborhood cooperative hysteretic DQN (NC-HDQN) is proposed. Specifically, for each NC-HDQN agent in the NMG, first, the framework analyses correlation degrees with their connected neighbors and weighs observations and rewards by these correlations. Second, NC-HDQN agents independently optimize their strategies on the weighted information using hysteretic DQN (HDQN), which is designed to learn optimal joint strategies in cooperative multiagent games. Third, a rule-based NC-HDQN method and a Pearson correlation coefficient based NC-HDQN method, i.e., empirical NC-HDQN (ENC-HDQN) and Pearson NC-HDQN (PNC-HDQN), respectively, are designed. The first method maps the correlation degree between two connected agents according to vehicle numbers on roads between the two agents. In contrast, the second method uses the Pearson correlation coefficient to calculate the correlation degree adaptively. Our methods are empirically evaluated in both a synthetic scenario and two real-world traffic scenarios and give better performances in almost every standard test metric for ATSC.

Leveraging autonomous vehicles in mixed-autonomy traffic networks with reinforcement learning-controlled intersections

ABSTRACT Development of new approaches to adaptive traffic signal control has received significant attention; an example is the reinforcement learning (RL), where training and implementation of an RL agent can allow adaptive signal control in real time, considering the agent’s past experiences. Furthermore, autonomous vehicle (AV) technology has shown promise to enhancing the traffic mobility at highways and intersections. In this paper, delayed action deep Q-learning is developed for a vehicle network with signalized intersections to control the signal phase. A model predictive control (MPC) scheme is proposed to allow AVs to adapt their speed. Several case studies that consider mixed autonomy are examined aiming at reducing network traffic and fuel consumption in the traffic network with multiple intersections. Simulation studies reveal that even with a few AVs in the network, the waiting time, fuel consumption, and the number of stop-and-go movements are significantly reduced, while the travel time is increased.

Real-Time multi-objective optimization of safety and mobility at signalized intersections

ABSTRACT Adaptive Traffic Signal Control (ATSC) is becoming a popular dynamic traffic management technique, especially with the emerging connected vehicles (CVs) technology. ATSC algorithms have been extensively considered in the literature for enhancing traffic mobility at signalized intersections. However, improving safety has rarely been used as an objective in existing ATSC algorithms. To fill this gap, this paper proposes a multi-criteria reinforcement learning based ATSC algorithm with two optimization objectives: real-time safety and mobility. The algorithm was trained on both objectives using traffic simulation. The safety objective was considered using extreme value theory (EVT) real-time crash risk evaluation models. Reducing the total intersection delay was the mobility objective. Different weights were considered in the training to account for both objectives simultaneously. The performance of the trained algorithm was then validated using real-world video data. Results show that the proposed multi-objective algorithm can improve both safety and mobility even under lower weights.

Adaptive Traffic Signal Control With Deep Reinforcement Learning and High Dimensional Sensory Inputs: Case Study and Comprehensive Sensitivity Analyses

Despite the constant rise in global urban populations and subsequent rise in transportation demand, significant expansion of infrastructure has been hampered by the constraints of space, cost, and environmental concerns. Therefore, optimizing the efficiency of existing infrastructure is becoming increasingly important. Adaptive traffic signal controllers aim to provide demand-responsive strategies to minimize motorists’ delay and achieve higher throughput at signalized intersections. With the advent of new sensory technologies and more intelligent control methods, the contribution of this paper is an adaptive traffic signal controller able to receive un-preprocessed high-dimensional sensory information such as GPS traces from connected vehicles and self-learn to minimize intersection delays. We use deep neural networks to operate directly on detailed sensory inputs and feed them into a reinforcement learning-based optimal control agent. The integration of these two components is known as deep learning. Using deep learning, we achieve two goals: (1) We eliminate the need for handcrafting a feature extraction process such as determining queue lengths, which is challenging and location-specific, and (2) we achieve better performance and faster training times compared to conventional tabular reinforcement learning approaches. We test our proposed controller against a tabular reinforcement learning agent, a reinforcement learning agent with a fully-connected Neural Network as a function approximator, and a state-of-practice, actuated traffic signal controller.

Real-Time Adaptive Traffic Signal Control in a Connected and Automated Vehicle Environment: Optimisation of Signal Planning with Reinforcement Learning under Vehicle Speed Guidance.

Adaptive traffic signal control (ATSC) is an effective method to reduce traffic congestion in modern urban areas. Many studies adopted various approaches to adjust traffic signal plans according to real-time traffic in response to demand fluctuations to improve urban network performance (e.g., minimise delay). Recently, learning-based methods such as reinforcement learning (RL) have achieved promising results in signal plan optimisation. However, adopting these self-learning techniques in future traffic environments in the presence of connected and automated vehicles (CAVs) remains largely an open challenge. This study develops a real-time RL-based adaptive traffic signal control that optimises a signal plan to minimise the total queue length while allowing the CAVs to adjust their speed based on a fixed timing strategy to decrease total stop delays. The highlight of this work is combining a speed guidance system with a reinforcement learning-based traffic signal control. Two different performance measures are implemented to minimise total queue length and total stop delays. Results indicate that the proposed method outperforms a fixed timing plan (with optimal speed advisory in a CAV environment) and traditional actuated control, in terms of average stop delay of vehicle and queue length, particularly under saturated and oversaturated conditions.

Adaptive Traffic Signal Control Based on Neural Network Prediction of Weighted Traffic Flow

Adaptive Traffic Signal Control Based on Neural Network Prediction of Weighted Traffic Flow

An Intelligent IoT Based Traffic Light Management System: Deep Reinforcement Learning

Traffic is one of the indispensable problems of modern societies, which leads to undesirable consequences such as time wasting and greater possibility of accidents. Adaptive Traffic Signal Control (ATSC), as a key part of Intelligent Transportation Systems (ITS), plays a key role in reducing traffic congestion by real-time adaptation to dynamic traffic conditions. Moreover, these systems are integrated with Internet of Things (IoT) devices. IoT can lead to easy implementation of traffic management systems. Recently, the combination of Artificial Intelligence (AI) and the IoT has attracted the attention of many researchers and can process large amounts of data that are suitable for solving complex real-world problems about traffic control. In this paper, we worked on the real-world scenario of Shiraz City, which currently does not use any intelligent method and works based on fixed-time traffic signal scheduling. We applied IoT approaches and AI techniques to control traffic lights more efficiently, which is an essential part of the ITS. Specifically, sensors such as surveillance cameras were used to capture real-time traffic information for the intelligent traffic signal control system. In fact, an intelligent traffic signal control system is provided by utilizing distributed Multi-Agent Reinforcement Learning (MARL) and applying the traffic data of adjacent intersections along with local information. By using MARL, our goal was to improve the overall traffic of six signalized junctions of Shiraz City in Iran. We conducted numerical simulations for two synthetic intersections by simulated data and for a real-world map of Shiraz City with real-world traffic data received from the transportation and municipality traffic organization and compared it with the traditional system running in Shiraz. The simulation results show that our proposed approach performs more efficiently than the fixed-time traffic signal control scheduling implemented in Shiraz in terms of average vehicle queue lengths and waiting times at intersections.

A Distributed Multi-Agent Reinforcement Learning With Graph Decomposition Approach for Large-Scale Adaptive Traffic Signal Control

With the emerging connected-vehicle technologies and smart roadways, the need for intelligent adaptive traffic signal controls (ATSC) is more than ever before. This paper first proposes an Accumulated Exponentially Weighted Waiting Time-based Adaptive Traffic Signal Control (AEWWT-ATSC) model to calculate priorities of roadways for signal scheduling. As the size of the traffic network grows, it adds great complexities and challenges to computational efficiencies. Considering this, we propose a novel Distributed Multi-agent Reinforcement Learning (DMARL) with a graph decomposition approach for large-scale ATSC problems. The decomposition clusters intersections by the level of connectivity (LoC), defined by the average residual capacities (ARC) between connected intersections, enabling us to train subgraphs instead of the entire network in a synchronized way. The problem is formulated as a Markov Decision Process (MDP), and the Double Dueling Deep Q Network with Prioritized Experience Replay is utilized to solve it. Under the optimal policy, the agents can select the optimal signal durations to minimize the waiting time and queue size. In evaluation, we show the superiority of the AEWWT-ATSC based RL methods in different densities and demonstrate the DMARL with a graph decomposition approach on a large graph in Manhattan, NYC. The approach is generic and can be extended to various types of use cases.

Dynamic capacity estimation of mixed traffic flows with application in adaptive traffic signal control

Intersection control plays a vital role in influencing transportation efficiency inside urban areas. Connected and Automated Vehicle (CAV) technology enables frequent traffic information sharing through vehicular networks, which emerges as a promising way to reduce vehicle traveling time and improve intersection capacity. Meanwhile, with the gradual deployment of CAVs, the traditional traffic flow consisting of human driven vehicles will evolve into a mixed traffic flow composed of traffic participants with differing intelligence capabilities. Considering the evolution towards the mixed traffic environment, we propose a dynamic capacity-aware traffic signal control method: max-pressure for mixed traffic flow (MPMF) traffic signal controller. Firstly, the lane capacity is modeled and approximated based on the saturation flow rate in mixed traffic flow considering the penetration rate of CAVs. Then the dynamic capacity is involved in calculating pressure in the max-pressure method to indicate the importance of a path in mixed traffic flow. A modification strategy for the max-pressure input value is also proposed. An isolated intersection scenario was simulated first to assess the proposed method locally. A multi-intersection network experiment was also conducted to verify the network-level performance of the proposed MPMF method. Comparative results between the proposed MPMF method, the classic max-pressure control, and the existing fixed time control method demonstrate that the MPMF can effectively improve the performance of intersections and be suitable for the multi-intersection road network.

An inductive heterogeneous graph attention-based multi-agent deep graph infomax algorithm for adaptive traffic signal control

Adaptive traffic signal control (ATSC) facilitates alleviating traffic congestion. Multi-agent deep reinforcement learning (MDRL) is a new promising algorithm for ATSC, and Graph Neural Networks (GNNs) further promote its learning ability. However, there are some drawbacks in the state-of-the-art MDRL algorithms. (1) These algorithms cannot effectively fuse diverse heterogeneous information of traffic networks due to adopting homogeneous GNNs; (2) These algorithms cannot be effectively trained due to merely adopting MDRL loss functions. In this paper, we propose an Inductive Heterogeneous graph Attention-based Multi-agent Deep Graph Infomax (IHA-MDGI) algorithm for ATSC. The proposed IHA-MDGI algorithm conducts both feature fusion via a proposed Inductive Heterogeneous graph Attention (IHA) algorithm and training via a proposed Multi-agent Deep Graph Infomax (MDGI) framework. Specifically, (1) Unlike the algorithms which adopt homogeneous GNNs, in the IHA algorithm, heterogeneous GNNs are designed to fuse both heterogeneous structural information and heterogeneous features of traffic networks, which aims to acquire heterogeneous information embeddings of traffic networks. (2) In the MDGI framework, the acquired embeddings are used to calculate the signal-control policies and Q-value for each agent, and then a mutual-information loss function is designed, which combines with the MDRL loss function to jointly train the whole algorithm. The designed mutual-information loss function focuses on maximizing mutual information between input (i.e., heterogeneous information embeddings) and output (i.e., Q-value), which can produce cooperative signal-control policies and maximize Q-value. We conduct the experiments in both real-traffic and synthetic-traffic networks under the time-varying traffic flows, and the results demonstrate that IHA-MDGI algorithm outperforms the state-of-the-art MDRL algorithms about multiple metrics.

Priority-Based Adaptive Traffic Signal Control System for Smart Cities

Priority-Based Adaptive Traffic Signal Control System for Smart Cities

Multi-Agent Transfer Reinforcement Learning With Multi-View Encoder for Adaptive Traffic Signal Control

Multi-agent reinforcement learning (MARL) based methods for adaptive traffic signal control (ATSC) have shown promising potentials to solve the heavy traffic problems. The existing MARL methods adopt centralized or distributed strategies. The former only models the environment as an agent and suffers from the exponential growth of action and state space. The latter extends the independent reinforcement learning methods, such as DQN, to multiple interactions directly or propagates information, such as state and policy, without taking their qualities into account. In this paper, we propose a multi-agent transfer reinforcement learning method to enhance the performance of MARL for ATSC, which is termed as multi-agent transfer soft actor-critic with the multi-view encoder (MT-SAC). The MT-SAC combines centralized and distributed strategies. In MT-SAC, we propose a multi-view state encoder and a transfer learning paradigm with guidance. The encoder processes input states from multiple perspectives and uses an attention mechanism to weigh the neighborhood information. While the paradigm enables the agents to handle different conditions for improving generalization abilities by transfer learning. Experimental studies on different scale road networks show that the MT-SAC outperforms the state-of-the-art algorithms and makes the traffic signal controllers more collaborative and robust.

A Novel Adaptive Traffic Signal Control Based on Cloud/Fog/Edge Computing

A Novel Adaptive Traffic Signal Control Based on Cloud/Fog/Edge Computing