Traffic Signal Control System Research Articles

Traffic Signal Control with Successor Feature-Based Deep Reinforcement Learning Agent

In this paper, we study the problem of traffic signal control in general intersections by applying a recent reinforcement learning technique. Nowadays, traffic congestion and road usage are increasing significantly as more and more vehicles enter the same infrastructures. New solutions are needed to minimize travel times or maximize the network capacity (throughput). Recent studies embrace machine learning approaches that have the power to aid and optimize the increasing demands. However, most reinforcement learning algorithms fail to be adaptive regarding goal functions. To this end, we provide a novel successor feature-based solution to control a single intersection to optimize the traffic flow, reduce the environmental impact, and promote sustainability. Our method allows for flexibility and adaptability to changing circumstances and goals. It supports changes in preferences during inference, so the behavior of the trained agent (traffic signal controller) can be changed rapidly during the inference time. By introducing the successor features to the domain, we define the basics of successor features, the base reward functions, and the goal preferences of the traffic signal control system. As our main direction, we tackle environmental impact reduction and support prioritized vehicles’ commutes. We include an evaluation of how our method achieves a more effective operation considering the environmental impact and how adaptive it is compared to a general Deep-Q-Network solution. Aside from this, standard rule-based and adaptive signal-controlling technologies are compared to our method to show its advances. Furthermore, we perform an ablation analysis on the adaptivity of the agent and demonstrate a consistent level of performance under similar circumstances.

A Review of Different Components of the Intelligent Traffic Management System (ITMS)

Traffic congestion is a serious challenge in urban areas. So, to address this challenge, the intelligent traffic management system (ITMS) is used to manage traffic on road networks. Managing traffic helps to focus on environmental impacts as well as emergency situations. However, the ITMS system has many challenges in analyzing scenes of complex traffic. New technologies such as computer vision (CV) and artificial intelligence (AI) are being used to solve these challenges. As a result, these technologies have made a distinct identity in the surveillance industry, particularly when it comes to keeping a constant eye on traffic scenes. There are many vehicle attributes and existing approaches that are being used in the development of ITMS, along with imaging technologies. In this paper, we reviewed the ITMS-based components that describe existing imaging technologies and existing approaches on the basis of their need for developing ITMS. The first component describes the traffic scene and imaging technologies. The second component talks about vehicle attributes and their utilization in existing vehicle-based approaches. The third component explains the vehicle’s behavior on the basis of the second component’s outcome. The fourth component explains how traffic-related applications can assist in the management and monitoring of traffic flow, as well as in the reduction of congestion and the enhancement of road safety. The fifth component describes the different types of ITMS applications. The sixth component discusses the existing methods of traffic signal control systems (TSCSs). Aside from these components, we also discuss existing vehicle-related tools such as simulators that work to create realistic traffic scenes. In the last section named discussion, we discuss the future development of ITMS and draw some conclusions. The main objective of this paper is to discuss the possible solutions to different problems during the development of ITMS in one place, with the help of components that would play an important role for an ITMS developer to achieve the goal of developing efficient ITMS.

Android-based System for Intelligent Traffic Signal Control and Emergency Call Functionality

The aim of the system was to utilize technology to save lives in India by addressing the challenge of transporting patients during emergencies on congested roads. To tackle this problem, we developed two mobile apps - one for users and one for ambulance drivers. The user app requires signup and retrieves their address and location. Similarly, the ambulance driver app also retrieves their location. When the user presses the emergency button, a notification is sent to the ambulance driver via the app. The app then displays the shortest path, and traffic signals along the route are adjusted to facilitate a faster route to the hospital. Overall, our apps were created to aid accident victims in reaching the hospital promptly

Deep Reinforcement Learning Based Traffic Signal Control: A Comparative Analysis

More recently, the advancement of deep learning techniques enables reinforcement learning (RL) to handle high-dimensional decision-making problems. It brings increasing attention in transport areas. Some work has applied them to solve the intractable traffic signal control (TSC) problem and achieved promising results. However, very few research comprehensively investigates the impacts of key design elements, including state definitions, reward functions and deep reinforcement learning (DRL) methods, on TSC performance. To fill this research gap, this paper first selects commonly used design elements from existing literature. Then, we compare their learning stability and control performance at an isolated intersection under different scenarios via simulation experiments. The experimental results show that the quantitative state (e.g., the number of vehicles on lanes) and image-like state (e.g., vehicle position and speed) have no significant impact on the performance under different traffic demands. However, the impact of various reward functions on performance is vivid, especially under high traffic demand. Also, high-resolution vehicular network data may not be possible to improve control performance versus ordinary camera data. In addition, the value-based DRL algorithms outperform the policy-based algorithm and traditional TSC control methods. The findings of this research would provide insights and guidance for transport engineers to design an efficient DRL-based TSC system in a real traffic environment.

Backdoor attacks against deep reinforcement learning based traffic signal control systems

Backdoor attacks against deep reinforcement learning based traffic signal control systems

EMVLight: A multi-agent reinforcement learning framework for an emergency vehicle decentralized routing and traffic signal control system

Emergency vehicles (EMVs) play a crucial role in responding to time-critical calls such as medical emergencies and fire outbreaks in urban areas. Existing methods for EMV dispatch typically optimize routes based on historical traffic-flow data and design traffic signal pre-emption accordingly; however, we still lack a systematic methodology to address the coupling between EMV routing and traffic signal control. In this paper, we propose EMVLight, a decentralized reinforcement learning (RL) framework for joint dynamic EMV routing and traffic signal pre-emption. We adopt the multi-agent advantage actor–critic method with policy sharing and spatial discounted factor. This framework addresses the coupling between EMV navigation and traffic signal control via an innovative design of multi-class RL agents and a novel pressure-based reward function. The proposed methodology enables EMVLight to learn network-level cooperative traffic signal phasing strategies that not only reduce EMV travel time but also shortens the travel time of non-EMVs. Simulation-based experiments indicate that EMVLight enables up to a 42.6% reduction in EMV travel time as well as an 23.5% shorter average travel time compared with existing approaches.

Extensible prototype learning for real‐time traffic signal control

AbstractCongestion resolution continues to remain a challenge even though various signal control systems have been developed for traffic‐intersection control. To address this issue, reinforcement learning (RL)‐based approaches that focus on solving the associated data‐driven problems have been proposed. However, only a few methods have been developed and applied to dual‐ring traffic signal control systems. Therefore, we develop an RL‐based traffic signal control model for such a system to efficiently allocate the green interval in different oversaturation states of the conflicting phases. The proposed model employs a deep deterministic policy gradient algorithm to optimize the green value in the continuous action space. Further, we develop an extensible prototype learning framework for application to new intersections without additional transfer learning. The proposed model is validated based on morning peak hours in a simulation environment that reflects the actual intersection phase system and minimum green time constraints. The proposed model achieves an average 20% intersection delay reduction, compared with the fixed control method.

Traffic signal control in mixed traffic environment based on advance decision and reinforcement learning

Abstract Reinforcement learning-based traffic signal control systems (RLTSC) can enhance dynamic adaptability, save vehicle travelling time and promote intersection capacity. However, the existing RLTSC methods do not consider the driver's response time requirement, so the systems often face efficiency limitations and implementation difficulties. We propose the advance decision-making reinforcement learning traffic signal control (AD-RLTSC) algorithm to improve traffic efficiency while ensuring safety in mixed traffic environment. First, the relationship between the intersection perception range and the signal control period is established and the trust region state (TRS) is proposed. Then, the scalable state matrix is dynamically adjusted to decide the future signal light status. The decision will be displayed to the human-driven vehicles (HDVs) through the bi-countdown timer mechanism and sent to the nearby connected automated vehicles (CAVs) using the wireless network rather than be executed immediately. HDVs and CAVs optimize the driving speed based on the remaining green (or red) time. Besides, the Double Dueling Deep Q-learning Network algorithm is used for reinforcement learning training; a standardized reward is proposed to enhance the performance of intersection control and prioritized experience replay is adopted to improve sample utilization. The experimental results on vehicle micro-behaviour and traffic macro-efficiency showed that the proposed AD-RLTSC algorithm can simultaneously improve both traffic efficiency and traffic flow stability.

SMART TRAFFIC SIGNAL CONTROL SYSTEM FOR TWO INTER-DEPENDENT INTERSECTIONS IN AKURE, NIGERIA

The increasing growth in urban population and vehicular volume coupled with inefficient traffic management results in traffic congestion on road networks. In this work, a smart/intelligent traffic signal control system was developed for two inter-dependent intersections in Akure, Nigeria. The system developed in this work uses deep learning and computer vision techniques to estimate the density of traffic and uses this information to adaptively switch traffic signals based on the traffic density estimated. Simulation results show that in 30 minutes of simulation, 32 signal cycles can be achieved and 967 vehicles can move at these two inter-dependent intersections

Improving traffic signal control operations using proximal policy optimization

AbstractExisting traffic signal control systems present many limitations including fixed signal timing schemes, insufficient efficiency and flexibility, and difficulty in adapting to the changing traffic flows. In recent years, the development of deep reinforcement learning (RL) has shown great research potential and application prospects. This paper proposes an intersection signal control method based on the proximal policy optimization (PPO) method. Specifically, this paper uses the value vector representation method of traffic characteristics to encode the traffic state and then feeds the state encoding into the long short‐term memory (LSTM) network to obtain the signal phase output. Finally, to obtain the optimal signal control strategy, the PPO algorithm is utilized to train the neural network and adjust the signal phase. The proposed algorithm is benchmarked against other classic RL and adaptive signal control schemes in an experimental environment based on the traffic simulation software simulation of urban mobility (SUMO). Experimental results showed that the proposed algorithm greatly improves traffic efficiency. Specifically, the mean velocity of the vehicles increases by 45.09%, and the mean occupancy rate of each lane, the length of the longest jams during each step, and the mean halting duration dropped by 21.38%, 25.86%, and 12.94%, respectively.

(Retracted) Intelligent rail transit signal control system based on image processing technology

With the rapid development of the intelligent rail field, the intelligent rail traffic signal control system plays an increasingly important role, bringing greater convenience to people’s travel. Under normal circumstances, the trains are dispatched strictly according to the schedule, but in the daily operation of urban rail transit, the service is inevitably affected by many unpredictable uncertain factors. To ensure the safety of vehicles and smooth traffic, we proposed an intelligent rail traffic signal control system based on image processing technology. When the system detects that there are too many people waiting, it automatically adjusts the display time of the signal lights in real time according to the image, thus ensuring more time for passengers to get on the train, so as to realize the intelligent control of the track. The experimental results of our paper show that the accuracy of the improved vehicle target detection and recognition algorithm is more than 90%, which is higher than that of the algorithm without the improvement. Applying the proposed algorithm to image processing improved the recognition rate and accuracy. Through the test of the intelligent rail traffic signal control system based on image processing, it was found that the system effectively identified the number of people waiting in the lane and automatically adjusted the train closing time. The time required was only 1 s, and the responsiveness reached 99% to ensure the safety of vehicles and smooth traffic.

Simulation of Transit Signal Priority Using the NTCIP Architecture

Transit Signal Priority (TSP) is an important element of Bus Rapid Transit (BRT) that involves coordinated efforts between transit vehicle detection systems, traffic signal control systems, and communication technologies. Successful deployment of TSP requires thorough laboratory evaluation through simulation before field implementation. This paper presents the development and application of a simulation model specifically designed for the design and evaluation of TSP systems. The proposed simulation tool models in detail all the TSP components in accordance with the National Transportation Communications for ITS Protocol (NTCIP) standard for TSP systems. The study is intended to shed light on how the variety of TSP elements can be addressed in microscopic simulation in a structured and systematic fashion. Sample applications of the model on a real-life arterial corridor in California demonstrate its capabilities and features.

A DSS Framework for Advanced Traffic Signal Control System Investment Planning

This article presents ongoing research on the initial development of a decision support system (DSS) framework for integrated emergency vehicle preemption and transit priority system investment planning. A conceptual intelligent DSS framework provides a holistic framework to perform analytical assessments of integrated emergency vehicle preemption and transit priority systems. Three analytical tools are presented for incorporation into future DSS design: the first addresses the potential impact of transit travel time reduction on transit operating costs; the second addresses the potential impact of reduced emergency vehicle crashes at signalized intersections on fire and rescue operating costs; and the third integrates fuzzy sets concepts and multiattribute decision-making methods to rank order transit signal priority strategy alternatives at the intersection level.

An Intelligent IoT Based Traffic Light Management System: Deep Reinforcement Learning

Traffic is one of the indispensable problems of modern societies, which leads to undesirable consequences such as time wasting and greater possibility of accidents. Adaptive Traffic Signal Control (ATSC), as a key part of Intelligent Transportation Systems (ITS), plays a key role in reducing traffic congestion by real-time adaptation to dynamic traffic conditions. Moreover, these systems are integrated with Internet of Things (IoT) devices. IoT can lead to easy implementation of traffic management systems. Recently, the combination of Artificial Intelligence (AI) and the IoT has attracted the attention of many researchers and can process large amounts of data that are suitable for solving complex real-world problems about traffic control. In this paper, we worked on the real-world scenario of Shiraz City, which currently does not use any intelligent method and works based on fixed-time traffic signal scheduling. We applied IoT approaches and AI techniques to control traffic lights more efficiently, which is an essential part of the ITS. Specifically, sensors such as surveillance cameras were used to capture real-time traffic information for the intelligent traffic signal control system. In fact, an intelligent traffic signal control system is provided by utilizing distributed Multi-Agent Reinforcement Learning (MARL) and applying the traffic data of adjacent intersections along with local information. By using MARL, our goal was to improve the overall traffic of six signalized junctions of Shiraz City in Iran. We conducted numerical simulations for two synthetic intersections by simulated data and for a real-world map of Shiraz City with real-world traffic data received from the transportation and municipality traffic organization and compared it with the traditional system running in Shiraz. The simulation results show that our proposed approach performs more efficiently than the fixed-time traffic signal control scheduling implemented in Shiraz in terms of average vehicle queue lengths and waiting times at intersections.

On the Cybersecurity of Traffic Signal Control System With Connected Vehicles

Connected vehicle (CV) technology brings both opportunities and challenges to the traffic signal control (TSC) system. While safety and mobility performance could be greatly improved by adopting CV technologies, the connectivity between vehicles and transportation infrastructure may increase the risks of cyber threats. In the past few years, studies related to cybersecurity on the TSC systems were conducted. However, there still lacks a systematic investigation that provides a comprehensive analysis framework. In this study, our aim is to fill the research gap by proposing a comprehensive analysis framework for the cybersecurity problem of the TSC in the CV environment. With potential threats towards the major components of the system and their corresponding impacts on safety and efficiency analyzed, data spoofing attack is considered the most plausible and realistic attack approach. Based on this finding, different attack strategies and defense solutions are discussed. A case study is presented to show the impact of the data spoofing attacks towards a selected CV based TSC system and corresponding mitigation countermeasures. This case study is conducted on a hybrid security testing platform, with virtual traffic and a real V2X communication network. To the best of our knowledge, this is the first study to present a comprehensive analysis framework to the cybersecurity problem of the CV-based TSC systems.

PrivacySignal: Privacy-Preserving Traffic Signal Control for Intelligent Transportation System

A new trend of using deep reinforcement learning for traffic signal control has become a spotlight in the Intelligent Transportation System (ITS). However, the traditional intelligent traffic signal control system always collects and transmits vehicle information (e.g., vehicle location, speed, etc.) in the form of plaintext, which would result in the leakage of commuters’ privacy and thus bring unnecessary troubles to users. In this paper, we propose a privacy-preserving traffic signal control for an intelligent transportation system (PrivacySignal). It relies on the existing road facilities to achieve the privacy of commuters, which guarantees the practicality of the system. Real-time decision-making and confidentiality of the system can be achieved simultaneously via the design of a series of secure and efficient interactive protocols, that are based on additive secret sharing, to perform the deep <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$Q$ </tex-math></inline-formula> -network (DQN). Moreover, the security of PrivacySignal is testified, meanwhile, the system effectiveness, and the overall efficiency of PrivacySignal is demonstrated through theoretical analysis and simulation experiments. Compared with the existing privacy-preserving schemes of the intelligent traffic signal, PrivacySignal provides a general DQN based privacy-preserving traffic signal control strategy architecture with high efficiency and low-performance loss.

Priority-Based Adaptive Traffic Signal Control System for Smart Cities

Priority-Based Adaptive Traffic Signal Control System for Smart Cities

Deep reinforcement learning for transit signal priority in a connected environment

Transit signal priority (TSP) is an effective measure to reduce traffic congestion and improve bus efficiency in metropolises. The connected vehicle technology and reinforcement learning (RL) algorithms can respectively provide more detailed and accurate information and more robust algorithms to traffic signal control systems, to develop smarter TSP strategies. This paper proposes an extended Dueling Double Deep Q-learning with invalid action masking (eD3QNI) algorithm for TSP strategy in a connected environment. The algorithm considers multiple conflicting bus priority requests and the constraints on the traffic light and phase skipping rule, aiming to improve the person delay of buses. Its performance is evaluated by simulation for a single intersection with two traffic demands and random arrivals, schedule deviations, occupancies of buses. Results demonstrate that eD3QNI produces lower average person delay and schedule delay than fixed-time signal, active TSP strategies, and other common RL methods. It also shows that the invalid action masking (IAM) method is superior to the usual variable decision points (VDP) method in terms of high convergence speed, effective performance improvement, and application of domain knowledge on the RL algorithm. The penetration rates of connected buses do not affect the converging speed of the proposed method, and an environment with a higher penetration rate will show better performance. Moreover, under the proposed method, different specific reward functions can be incorporated as desired to realize different operational goals for the TSP strategies.

Multimodal Traffic Speed Monitoring: A Real-Time System Based on Passive Wi-Fi and Bluetooth Sensing Technology

Traffic speed is one of the critical indicators reflecting traffic status of roadway networks. The abnormality and sudden changes of traffic speed indicate the occurrence of traffic congestions, accidents, and events. Traffic control and management systems usually take the spatiotemporal variations of traffic speed as the critical evidence to dynamically adjust the traffic signal timing plan, broadcast traffic accidents, and form a management strategy. Meanwhile, transport is multimodal in most cities, including vehicles, pedestrians, and bicyclists. Traffic states of different traffic modes are usually used simultaneously as the significant input of advanced traffic control systems, e.g., multiobjective traffic signal control system, connected vehicles, and autonomous driving. In previous studies, Wi-Fi and Bluetooth passive sensing technology was demonstrated as an effective method for obtaining traffic speed data. However, there are some challenges that greatly affect the accuracy the estimated traffic speed, e.g., traffic mode uncertainty and the errors caused by sensors’ detection range. Thus, this study develops a real-time method for estimating the multimodal traffic speed of road networks covered by Wi-Fi and Bluetooth passive sensors. To address the two identified challenges, an algorithm is developed to correct the biased estimated traffic speed based on the received signal strength indicator of Wi-Fi and Bluetooth signals, and a novel semisupervised Possibilistic Fuzzy <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$C$ </tex-math></inline-formula> -Means clustering algorithm is proposed for identifying traffic modes of Wi-Fi and Bluetooth device owners. The performance of the proposed algorithms is evaluated by comparing with the selected baseline algorithms. The experimental results indicate the superiority of the proposed algorithm. The proposed method of this study can provide accurate and real-time multimodal traffic speed information for supporting traffic control and management, and, thus, improving the operational performance of the whole road network.

Traffic Signal Control Using End-to-End Off-Policy Deep Reinforcement Learning

An efficient transportation system can substantially benefit our society, but road intersections have always been among the major traffic bottlenecks leading to traffic congestion. Appropriate traffic signal timing adapted to real-time traffic may help mitigate such traffic congestion. However, most existing traffic signal control methods require a huge amount of road information, such as vehicle positions. In this paper, we focus on a particular road intersection and aim to minimize the average waiting time. We propose a traffic signal control (TSC) system based on an end-to-end off-policy deep reinforcement learning (deep RL) agent with background removal residual networks. The agent takes real-time images at the road intersection as input. Upon sufficient training, the agent can perform (near-) optimal traffic signaling based on real-time traffic conditions. We conduct experiments on different intersection scenarios and compare various TSC methods. The experimental results show that our end-to-end deep RL approach can adapt to the dynamic traffic based on the traffic images and outperforms other TSC methods.