Intelligent Traffic Signal Control Research Articles

A multi-agent reinforcement learning based approach for intelligent traffic signal control

A multi-agent reinforcement learning based approach for intelligent traffic signal control

Optimizing Urban Mobility: A Comprehensive Study on Intelligent Traffic Signal Control Systems

This paper explores the application of Adaptive Traffic Signal Control (ATSC) systems integrated with Information and Communication Technology (ICT) and the Internet of Things (IoT) to improve urban traffic management. ATSC systems utilize real-time traffic data from various sensors and cameras to adjust signal timings dynamically, aiming to reduce congestion and enhance flow efficiency. The study highlights how these systems process and react to live traffic conditions, discussing the implementation of machine learning models and optimization algorithms that refine signal timing strategies based on traffic density and flow patterns. Challenges like data accuracy, system integration, cybersecurity, and environmental impacts on sensor performance are examined, alongside the potential for improving urban mobility and reducing environmental effects through smarter traffic management solutions. The paper concludes by emphasizing the importance of continuous advancements in technology and infrastructure to support the scalability and effectiveness of ATSC systems in modern cities.

A Comparative Study of Traffic Signal Control Based on Reinforcement Learning Algorithms

In recent years, the increasing production and sales of automobiles have led to a notable rise in congestion on urban road traffic systems, particularly at ramps and intersections with traffic signals. Intelligent traffic signal control represents an effective means of addressing traffic congestion. Reinforcement learning methods have demonstrated considerable potential for addressing complex traffic signal control problems with multidimensional states and actions. In this research, the team propose Q-learning and Deep Q-Network (DQN) based signal control frameworks that use variable phase sequences and cycle times to adjust the order and the duration of signal phases to obtain a stable traffic signal control strategy. Experiments are simulated using the traffic simulator Simulation of Urban Mobility (SUMO) to test the average speed and the lane occupancy rate of vehicles entering the ramp to evaluate its safety performance and test the vehicle’s traveling time to assess its stability. The simulation results show that both reinforcement learning algorithms are able to control cars in dynamic traffic environments with higher average speed and lower lane occupancy rate than the no-control method and that the DQN control model improves the average speed by about 10% and reduces the lane occupancy rate by about 30% compared to the Q-learning control model, providing a higher safety performance.

Live Intersection Data Acquisition for Traffic Simulators (LIDATS).

Real-time traffic signal acquisition and network transmission are essential components of intelligent transportation systems, facilitating real-time traffic monitoring, management, and analysis in urban environments. In this paper, we introduce a comprehensive system that incorporates live traffic signal acquisition, real-time data processing, and secure network transmission through a combination of hardware and software modules, called LIDATS. LIDATS stands for Live Intersection Data Acquisition for Traffic Simulators. The design and implementation of our system are detailed, encompassing signal acquisition hardware as well as a software platform that is used specifically for real-time data processing. The performance evaluation of our system was conducted by simulation in the lab, demonstrating its capability to reliably capture and transmit data in real time, and to effectively extract the relevant information from noisy and complex traffic data. Supporting a variety of intelligent transportation applications, such as real-time traffic flow management, intelligent traffic signal control, and predictive traffic analysis, our system enables remote data analysis and decisionmaking, providing valuable insights and enhancing the traffic efficiency while reducing the congestion in urban environments.

Emergency Vehicle Detection and Traffic Prevention Using Open CV

The proposed project focuses on the development of an innovative system for "Emergency Vehicle Detection and Traffic Prevention using OpenCV." The project aims to enhance road safety and expedite emergency response by leveraging computer vision techniques implemented through the OpenCV framework. The primary objective is to design a robust algorithm that can accurately identify emergency vehicles in real-time based on distinct visual features such as color, shape, and motion patterns. OpenCV, a powerful open-source computer vision library, will serve as the foundation for image processing and analysis. Upon successful detection of an emergency vehicle, the system will employ dynamic traffic prevention measures. This includes the activation of intelligent traffic signal control to ensure the smooth and rapid passage of the emergency vehicle through intersections. The system may also interface with connected infrastructure, such as smart traffic lights, to optimize overall traffic flow and minimize delays. The project aims to contribute to public safety and emergency response efficiency by providing a reliable and automated solution for prioritizing emergency vehicles on the road. Through the integration of OpenCV, the system will offer a versatile and scalable approach to emergency vehicle detection and traffic prevention, making it a valuable contribution to the field of intelligent transportation systems. Index terms Emergency Vehicle Detection, Traffic Prevention, OpenCV, Computer Vision Techniques, Real-time Vehicle Identification, Intelligent Traffic Signal Control, Dynamic Traffic Management, Road Safety Enhancement, Emergency Response Efficiency, Public Safety, Automated Traffic Prioritization, Intelligent Transportation Systems, Visual Feature Analysis, Smart Traffic Lights Integration, Automated Traffic Flow Optimization

KeyLight: Intelligent Traffic Signal Control Method Based on Improved Graph Neural Network

Graph neural network combined with reinforcement learning is one of the most effective traffic signal control methods. However, existing methods fail to pay enough attention to the key information, such as the traffic information of the downstream section in extracting state and the intersection’s own state in aggregating information from neighbor intersections. As a result, adverse reactions like unstable learning and limited performance occur frequently when agents and models focus on interfering information and useless information too much. In this article, we propose KeyLight, an intelligent traffic signal control method based on reinforcement learning by facilitating the attention of the learning algorithm and model to the key information that is usually ignored. In KeyLight, we design a new state representation NOV-LADLE and introduce residual connection in the graph neural network to highlight the importance of the intersection’s state. Experiments show that, in the case of comparable throughput, the proposed KeyLight has been greatly improved and enhanced in performance. Especially, the average travel time can be increased by up to 23.44% on the real-world dataset.

MARLens: Understanding Multi-Agent Reinforcement Learning for Traffic Signal Control Via Visual Analytics.

The issue of traffic congestion poses a significant obstacle to the development of global cities. One promising solution to tackle this problem is intelligent traffic signal control (TSC). Recently, TSC strategies leveraging reinforcement learning (RL) have garnered attention among researchers. However, the evaluation of these models has primarily relied on fixed metrics like reward and queue length. This limited evaluation approach provides only a narrow view of the model's decision-making process, impeding its practical implementation. Moreover, effective TSC necessitates coordinated actions across multiple intersections. Existing visual analysis solutions fall short when applied in multi-agent settings. In this study, we delve into the challenge of interpretability in multi-agent reinforcement learning (MARL), particularly within the context of TSC. We propose MARLens, a visual analytics system tailored to understand MARL-based TSC. Our system serves as a versatile platform for both RL and TSC researchers. It empowers them to explore the model's features from various perspectives, revealing its decision-making processes and shedding light on interactions among different agents. To facilitate quick identification of critical states, we have devised multiple visualization views, complemented by a traffic simulation module that allows users to replay specific training scenarios. To validate the utility of our proposed system, we present three comprehensive case studies, incorporate insights from domain experts through interviews, and conduct a user study. These collective efforts underscore the feasibility and effectiveness of MARLens in enhancing our understanding of MARL-based TSC systems and pave the way for more informed and efficient traffic management strategies.

Enhancing traffic signal control with composite deep intelligence

Traffic signal control has always been a hot topic in the field of intelligent transportation. With the increasing complexity of urban traffic conditions due to urbanization, how to develop effective scheduling strategies to adapt to the changing traffic demands has become a key problem in current intelligent transportation. In light of this, this paper focuses on the traffic signal control problem at intersections and proposes a composite intelligent traffic signal control model based on heterogeneous graph neural networks with dual attention mechanisms and deep reinforcement learning. For the first time, the model incorporates the dual attention mechanism in graph neural networks into the traffic signal control, integrating graph neural networks with deep reinforcement learning techniques and traffic intersection scenarios. This allows for the construction of traffic condition models and the scheduling control of traffic resources, catering to the perception and decision-making needs in complex traffic environments. Firstly, the graph relationship representation of intersection resources is established, constructing the graph information structure for traffic flow and signal states. Then, a heterogeneous graph neural network is designed, incorporating both node-level and semantic-level dual attention mechanisms to characterize the traffic state and explore the relationships, enabling the extraction of explicit and implicit information in traffic intersections. Lastly, a deep reinforcement learning algorithm that combines Double Deep Q-Network (DDQN) and Dueling DQN is implemented to improve the algorithm's generalization and execution efficiency, enhancing the adaptability and stability of traffic signal scheduling in complex environments. Simulation tests are conducted on the SUMO simulation platform using real-world application datasets. Compared to four other similar traffic control model, the proposed model demonstrates performance advantages of more than 13% in terms of average reward, average delay, queue length, and waiting time. This validates the effectiveness of the proposed model.

Reinforcement Learning-Based Intelligent Traffic Signal Control Considering Sensing Information of Railway

Reinforcement Learning-Based Intelligent Traffic Signal Control Considering Sensing Information of Railway

Improved Deep Reinforcement Learning for Intelligent Traffic Signal Control Using ECA_LSTM Network

Reinforcement learning is one of the most widely used methods for traffic signal control, but the method experiences issues with state information explosion, inadequate adaptability to special scenarios, and low security. Therefore, this paper proposes a traffic signal control method based on the efficient channel attention mechanism (ECA-NET), long short-term memory (LSTM), and double Dueling deep Q-network (D3QN), which is EL_D3QN. Firstly, the ECA-NET and LSTM module are included in order to lessen the state space’s design complexity, improve the model’s robustness, and adapt to various emergent scenarios. As a result, the cumulative reward is improved by 27.9%, and the average queue length, average waiting time, and CO2 emissions are decreased by 15.8%, 22.6%, and 4.1%, respectively. Next, the dynamic phase interval tgap is employed to enable the model to handle more traffic conditions. Its cumulative reward is increased by 34.2%, and the average queue length, average waiting time, and CO2 emissions are reduced by 19.8%, 30.1%, and 5.6%. Finally, experiments are carried out using various vehicle circumstances and unique scenarios. In a complex environment, EL_D3QN reduces the average queue length, average waiting time, and CO2 emissions by at least 13.2%, 20.2%, and 3.2% compared to the four existing methods. EL_D3QN also exhibits good generalization and control performance when exposed to the traffic scenarios of unequal stability and equal stability. Furthermore, even when dealing with unique events like a traffic surge, EL_D3QN maintains significant robustness.

Intelligent traffic signal controller for heterogeneous traffic using reinforcement learning

A traffic signal controller is an essential part of a signalized intersection to alleviate congestion and pollution by ensuring safety. However, the available research solutions are focused on homogeneous traffic scenarios, whereas heterogeneous traffic is the reality in most countries. Hence, a traffic signal control scheme suitable for heterogeneous traffic conditions is proposed in the current study using Reinforcement Learning. A novel reward function with an objective to reduce the traffic residual is defined and a combination of exploration and exploitation optimal policy is applied which made the system learn quickly. The proposed scheme can choose the appropriate phase sequence with optimal signal lengths based on traffic demand on each approaching road. The simulation results proved that the proposed model is well-suited for heterogeneous traffic conditions and its performance against the actuated traffic signal controller is significant in reducing the green time wastage and mean waiting time at the intersection.

Intelligent Traffic Signal Control using RF Technology for Emergency Vehicles

The transportation system in our nation has been greatly impacted by the problem of traffic congestion. This leads to a number of complications or problems, especially when there are emergencies in the heavily trafficked lanes at traffic lights. To address these challenges, a traffic light management system is built in. Based on an RFID module, this system was designed to operate when it received a signal from an emergency vehicle. The Programmable Arduino nano micro controller was used to regulate the LEDs used in the traffic signals. The system's use of LEDs makes it easier for emergency vehicles to manoeuvre through traffic. As a result, the project's analysis and implementation of the traffic signal control system for emergency vehicles were successful.

Dynamic traffic signal control using mean field multi‐agent reinforcement learning in large scale road‐networks

AbstractMulti‐agent reinforcement learning has played an increasingly important role in intelligent traffic signal control due to its self‐learning ability. However, existing algorithms only focus on signal timing mechanism design while ignoring the exponential growth of the joint action dimension as the number of intersections increases, which will ultimately face the learning difficulty. In this paper, traditional traffic methods are introduced into MARL to flexibly determine the phase and duration of each intersection. The proposed MARL algorithm based on mean field theory has the ability to convert a large number of agents to approximately binary interaction, which can effectively reduce the dimension of joint action space in multi‐agent environment and learn in a robust process. Besides, to improve the performance of traditional traffic methods, the recurrent neural network (RNN) and an improved Webster's formula with revised parameters are combined to dynamically determine the phase duration according to the historical volume of traffic flow. The simulation results indicate that the proposed algorithm shows superior scalability compared to baseline methods and has great potential to be applied in the large scale road‐networks scenario.

A citywide TD‐learning based intelligent traffic signal control for autonomous vehicles: Performance evaluation using SUMO

AbstractAn autonomous vehicle can sense its environment and operate without human involvement. Its adequate management in an intelligent transportation system could significantly reduce traffic congestion and overall travel time in a network. Adaptive traffic signal controller (ATSC) based on multi‐agent systems using state‐action‐reward‐state‐action (SARSA ()) are well‐known state‐of‐the‐art models to manage autonomous vehicles within urban areas. However, this study found inefficient weights updating mechanisms of the conventional SARSA () models. Therefore, it proposes a Gaussian function to regulate the eligibility trace vector's decay mechanism effectively. On the other hand, an efficient understanding of the state of the traffic environment is crucial for an agent to take optimal actions. The conventional models feed the state values to the agents through the MinMax normalization technique, which sometimes shows less efficiency and robustness. So, this study suggests the MaxAbs scaled state values instead of MinMax to address the problem. Furthermore, the combination of the A‐star routing algorithm and proposed model demonstrated a good increase in performance relatively to the conventional SARSA ()‐based routing algorithms. The proposed model and the baselines were implemented in a microscopic traffic simulation environment using the SUMO package over a complex real‐world‐like ‐intersections network to evaluate their performance. The results showed a reduction of the vehicle's average total waiting time and total stops by a mean value of % and % compared to the considered baselines. Also, the A‐star combined with the proposed controller outperformed the conventional approaches by increasing the vehicle's average trip speed by %.

Traffic Safety Oriented Multi-Intersection Flow Prediction Based on Transformer and CNN

Intelligent traffic signal control is one of the important means to ensure traffic safety. Effective signal control can make traffic flow fast and smooth, which first needs current and future traffic information. As the control of one intersection may affect adjacent intersections, this paper proposes to predict future traffic flow with consideration of multi-intersections. It can dynamically improve traffic conditions and reduce traffic congestion. Based on various nonlinear spatial relationships at correlated multi-intersections and the potential time-dependent relationship in traffic volume, a traffic flow prediction method named CNNformer which combines transformer with CNN, is proposed. The convolution neural network (CNN) and transformer are used to extract the spatial and temporal features of correlated multiple intersections. The learnable time code is embedded into transformer’s location code, and the location information and time information are injected into the model to help it learn the time characteristics of traffic volume. This study also considers the impact of cyclical traffic flow pattern and proposes CNNformer+. In the method, all of the data from the previous time window, as well as the data from the prior week and month, are correspondingly entered into the network. Finally, the output is generated through average pooling, realizing the learning of cyclical traffic flow characteristics. In the experiment, CNNformer+ and the state-of-the-art traffic flow prediction methods are compared using simulated data. The results show that the proposed model outperforms the existing models in prediction accuracy and efficiency.

Deep Reinforcement Q-Learning for Intelligent Traffic Signal Control with Partial Detection

Intelligent traffic signal controllers, applying DQN algorithms to traffic light policy optimization, efficiently reduce traffic congestion by adjusting traffic signals to real-time traffic. Most propositions in the literature however consider that all vehicles at the intersection are detected, an unrealistic scenario. Recently, new wireless communication technologies have enabled cost-efficient detection of connected vehicles by infrastructures. With only a small fraction of the total fleet currently equipped, methods able to perform under low detection rates are desirable. In this paper, we propose a deep reinforcement Q-learning model to optimize traffic signal control at an isolated intersection, in a partially observable environment with connected vehicles. First, we present the novel DQN model within the RL framework. We introduce a new state representation for partially observable environments and a new reward function for traffic signal control, and provide a network architecture and tuned hyper-parameters. Second, we evaluate the performances of the model in numerical simulations on multiple scenarios, in two steps. At first in full detection against existing actuated controllers, then in partial detection with loss estimates for proportions of connected vehicles. Finally, from the obtained results, we define thresholds for detection rates with acceptable and optimal performance levels. The source code implementation of the model is available at: https://github.com/romainducrocq/DQN-ITSCwPD

A Resilient Intelligent Traffic Signal Control Scheme for Accident Scenario at Intersections via Deep Reinforcement Learning

Deep reinforcement learning methods have shown promising results in the development of adaptive traffic signal controllers. Accidents, weather conditions, or special events all have the potential to abruptly alter the traffic flow in real life. The traffic light must take immediate and appropriate action based on a reasonable understanding of the environment. In this way, traffic congestion would be prevented. In this paper, we develop a reliable controller for such a highly dynamic environment and investigate the resilience of these controllers to a variety of environmental disruptions, such as accidents. In this method, the agent is provided with a complete understanding of the environment by discretizing the intersection and modifying the state space. The proposed algorithm is independent of the location and time of accidents. If the location of the accident changes, the agent does not need to be retrained. The agent is trained using deep Q-learning and experience replay. The model is evaluated in the traffic microsimulator SUMO. The simulation results demonstrate that the proposed method is effective at shortening queues when there is disruption.

Android-based System for Intelligent Traffic Signal Control and Emergency Call Functionality

The aim of the system was to utilize technology to save lives in India by addressing the challenge of transporting patients during emergencies on congested roads. To tackle this problem, we developed two mobile apps - one for users and one for ambulance drivers. The user app requires signup and retrieves their address and location. Similarly, the ambulance driver app also retrieves their location. When the user presses the emergency button, a notification is sent to the ambulance driver via the app. The app then displays the shortest path, and traffic signals along the route are adjusted to facilitate a faster route to the hospital. Overall, our apps were created to aid accident victims in reaching the hospital promptly

A Gain With No Pain: Exploring Intelligent Traffic Signal Control for Emergency Vehicles

For the emergency response, every second counts. Intersections are prone to congestion, which greatly hinders the fast response of emergency vehicles. Although emergency vehicles possess the privilege to run a red light, it can be unsafe, and a congested intersection will prevent the exercise of this privilege. When an emergency vehicle arrives, the greedy preemption scheme offers a green signal promptly until it leaves the intersection. This guarantees a fast emergency response in most cases. However, this scheme will lead to an adverse impact on vehicles of conflicting directions and may not work when there are other emergency vehicles traveling from conflicting directions simultaneously. Employing deep reinforcement learning techniques, recent studies have shown promising results for traffic signal control. In this work, we deliver an early attempt to control the traffic signal for emergency vehicles through deep reinforcement learning, which ensures an expeditious emergency response in various scenarios and alleviates the negative influence on the traffic efficiency of conflicting directions. We conduct realistic simulations using traffic data in a real-world network with multiple intersections on different testing parameters. The results verify the feasibility and effectiveness of our model and indicate that our method notably outperforms the other five baseline methods in terms of various performance metrics.

An Intelligent IoT Based Traffic Light Management System: Deep Reinforcement Learning

Traffic is one of the indispensable problems of modern societies, which leads to undesirable consequences such as time wasting and greater possibility of accidents. Adaptive Traffic Signal Control (ATSC), as a key part of Intelligent Transportation Systems (ITS), plays a key role in reducing traffic congestion by real-time adaptation to dynamic traffic conditions. Moreover, these systems are integrated with Internet of Things (IoT) devices. IoT can lead to easy implementation of traffic management systems. Recently, the combination of Artificial Intelligence (AI) and the IoT has attracted the attention of many researchers and can process large amounts of data that are suitable for solving complex real-world problems about traffic control. In this paper, we worked on the real-world scenario of Shiraz City, which currently does not use any intelligent method and works based on fixed-time traffic signal scheduling. We applied IoT approaches and AI techniques to control traffic lights more efficiently, which is an essential part of the ITS. Specifically, sensors such as surveillance cameras were used to capture real-time traffic information for the intelligent traffic signal control system. In fact, an intelligent traffic signal control system is provided by utilizing distributed Multi-Agent Reinforcement Learning (MARL) and applying the traffic data of adjacent intersections along with local information. By using MARL, our goal was to improve the overall traffic of six signalized junctions of Shiraz City in Iran. We conducted numerical simulations for two synthetic intersections by simulated data and for a real-world map of Shiraz City with real-world traffic data received from the transportation and municipality traffic organization and compared it with the traditional system running in Shiraz. The simulation results show that our proposed approach performs more efficiently than the fixed-time traffic signal control scheduling implemented in Shiraz in terms of average vehicle queue lengths and waiting times at intersections.