Real-world Traffic System Research Articles

Learning in practice: reinforcement learning-based traffic signal control augmented with actuated control

ABSTRACT Most Reinforcement Learning (RL) based Traffic Signal Control (TSC) models are trained in simulation platforms, which inevitably suffer from the performance degradation after deployment due to the mismatch between the traffic simulator and the real-world traffic system. Toward the real-world training of TSC agents, this study integrates the actuated control into the RL-based TSC framework as a defense mechanism, enabling the traffic system to remain resilient to suboptimal actions of agent during the real-world training process. The introduction of actuated control can significantly reduce the green time wastes due to unreasonable timing plans. The numerical results demonstrate that the proposed approach can reduce interferences to the traffic system operation during the real-world training of TSC agents, particularly at the early stage of agent learning, thereby promoting the practical deployment of RL-based TSC models. Meanwhile, agents can converge to better signal policies with the help of defense mechanism.

A feedback linearization approach for coordinated traffic flow management in highway systems

In this paper, a control solution to reduce congestion in highway traffic systems is presented. The aim is to produce a control strategy characterized by low computational cost, so that real-time implementation can be attained. The adopted model to describe traffic dynamics is the METANET model. A particular spatio-temporal derivative relationship, describing how control signals (ramp metering and variable speed limits) and disturbances effects propagate along the highway system, is highlighted in the paper. This relationship is the basis of a proposition providing the essential tool for relative degree calculation in generic highway systems. Utilizing this proposition, a feedback linearization-based control law is developed. The control design is completed by employing a linear MPC, which allows for complying with the physical constraints. The performance of the proposed method is evaluated by conducting comprehensive simulation studies, also considering a real-world traffic system. The computational costs are analyzed by comparing the developed methodology with a nonlinear MPC-based approach. Simulation evidence confirms that the proposed method can provide satisfactory solutions for coordinating RM and VSL in highway systems. Such solutions are compatible with real-time implementation.

Unified Automatic Control of Vehicular Systems With Reinforcement Learning

Emerging vehicular systems with increasing proportions of automated components present opportunities for optimal control to mitigate congestion and increase efficiency. There has been a recent interest in applying deep reinforcement learning (DRL) to these nonlinear dynamical systems for the automatic design of effective control strategies. Despite conceptual advantages of DRL being model-free, studies typically nonetheless rely on training setups that are painstakingly specialized to specific vehicular systems. This is a key challenge to efficient analysis of diverse vehicular and mobility systems. To this end, this article contributes a streamlined methodology for vehicular microsimulation and discovers high performance control strategies with minimal manual design. A variable-agent, multi-task approach is presented for optimization of vehicular Partially Observed Markov Decision Processes. The methodology is experimentally validated on mixed autonomy traffic systems, where fractions of vehicles are automated; empirical improvement, typically 15-60% over a human driving baseline, is observed in all configurations of six diverse open or closed traffic systems. The study reveals numerous emergent behaviors resembling wave mitigation, traffic signaling, and ramp metering. Finally, the emergent behaviors are analyzed to produce interpretable control strategies, which are validated against the learned control strategies. Note to Practitioners—As vehicular systems such as real-world traffic systems and robotic warehouses become increasingly automated, optimizing vehicle movements sees an increasing potential to reduce congestion and increase efficiency. For many vehicular systems, simulations of varying fidelity are commonly used for analysis and optimization without the need to deploy real vehicles. This article describes a unified and practical approach for optimal control of vehicles in arbitrary simulated vehicular systems while permitting partial automation, where the behavior of fractions of vehicles at given times can be modelled but not controlled. As illustrated by the diverse traffic systems considered in this article, the presented methodology emphasizes ease of application within any simulated vehicular system while minimizing manual efforts by the practitioner. The control inputs consist of local information around each automated vehicle, while the control outputs are commands for longitudinal acceleration and lateral lane change. Experimental results are presented for relatively small simulated traffic systems, though the methodology can be adapted to larger vehicular systems with minor modifications. Experimentally optimized behaviors provide insights to the practitioner which may assist in designing simplified and interpretable control strategies. Implementation in real-world systems depends on two requirements: 1) a reliable fallback mechanism for ensuring safety of vehicles, and 2) sufficient fidelity of the simulator for simulated behaviors to transfer. These requirements are under active research for traffic systems and may be practical in some robotic settings. To facilitate robust transfer of policies from simulated to real-world systems, future extensions of this work may inject additional randomization into simulation while reducing the unmodeled stochasticity of targeted real-world systems as much as possible.

Modeling Melburnians—Using the Koopman operator to gain insight into crowd dynamics

Describing and forecasting city traffic is challenging, given the array of factors influencing the movement of pedestrians and vehicles. Faced with this complexity, research has focused on machine learning as a way to capture spatio-temporal traffic patterns, based on past sensor data. While the methods can accurately forecast high-dimensional observations, it is often hard to explain the models and their predictions. In many cases, only the predictive performance of an otherwise black box model is evaluated. An accurate but explainable alternative is provided by the Koopman operator, an emerging methodology for data-driven identification of high-dimensional and nonlinear dynamical systems. In this study, we showcase the method by analyzing pedestrian traffic data from Melbourne, Australia. We formulate our computations in the Extended Dynamic Mode Decomposition framework, where we approximate the Koopman operator in a function basis computed with time delay embedding and the Diffusion Map algorithm. The model captures the distinct temporal patterns of 11 traffic sensors simultaneously. Because the dynamics become linear in the operator-theoretic perspective, we can decompose the model into its spectral components. Importantly, these components facilitate interpretation and can increase scientific understanding of the underlying system and the model’s operations. For the Melbourne data, we show that the spectral components connect to the underlying state space geometry and indicate the model’s stability over a prediction horizon. Our study showcases how the Koopman operator framework offers explainable and accurate data-driven predictions in a real-world traffic system. The results can easily be transferred to other traffic systems.

Auto-generation of centerline graphs from geometrically complex roadmaps of real-world traffic systems using hierarchical quadtrees for cellular automata simulations

This paper proposes a method for auto-generating centerline graphs from geometrically complex roadmaps of real-world traffic systems for cellular automata simulations, using hierarchical quadtrees. Our method is summarized as follows: First, we store the binary values of the monochrome image of the target roadmap (where one and zero represent the road and other areas, respectively) in a two-dimensional square map. Second, we recursively divide the square map into sub-leaves using a quadtree, until the sum of the values of pixels included inside each leaf becomes less than or equal to one. Third, we successively remove the distal leaves for which adjacent leaves have shallower depths. Fourth, we trace the remaining distal leaves of the tree using Morton’s space-filling curve, while selecting the leaves among those selected previously that preserve a certain distance as the nodes of the graph. Finally, each selected node searches the neighboring nodes, and these are stored as the edges of the graph. We demonstrate our method by generating a centerline graph from a complex roadmap of a real-world airport, and by performing a typical network analysis using Dijkstra’s method.

Threshold Queueing to Describe the Fundamental Diagram of Uninterrupted Traffic

Queueing because of congestion is an important aspect of road traffic. This paper provides a novel threshold queue that models the empirical shape of the fundamental diagram. In particular, we show that our threshold queue with two service phases captures the capacity drop that is eminent in the fundamental diagram of modern traffic. We use measurements on a Danish highway to illustrate that our threshold queue is indeed capable of capturing the fundamental diagram of real-world traffic systems. We furthermore indicate the modelling power of our threshold queue via a sensitivity study showing that our model is able to capture a wide range of shapes for the fundamental diagram. The online appendix is available at https://doi.org/10.1287/trsc.2018.0850 .

Development of a behaviorally induced system optimal travel demand management system

ABSTRACTThe basic design concept of most advanced traveler information systems (ATIS) is to present generic information to travelers, leaving travelers to react to the information in their own way. This “passive” way of managing traffic by providing generic traffic information makes it difficult to predict the outcome and may even incur an adverse effect, such as overreaction (also referred to as the herding effect). Active traffic and demand management (ATDM) is another approach that has received continual attention from both academic research and real-world practice, aiming to effectively influence people's travel demand, provide more travel options, coordinate between travelers, and reduce the need for travel. The research discussed in this article deals with how to provide users with a travel option that aims to minimize the marginal system impact that results from this routing. The goal of this research is to take better advantage of the available real-time traffic information provided by ATIS, to further improve the system level traffic condition from User Equilibrium (UE), or a real-world traffic system that is worse than UE, toward System Optimal (SO), and avoid passively managing traffic. A behaviorally induced, system optimal travel demand management model is presented to achieve this goal through incremental routing. Both analytical derivation and numerical analysis have been conducted on Tucson network in Arizona, as well as on the Capital Area Metropolitan Planning Organization (CAMPO) network in Austin, TX. The outcomes of both studies show that our proposed modeling framework is promising for improving network traffic conditions toward SO, and results in substantial economic savings.

Exploring the impact of a coordinated variable speed limit control on congestion distribution in freeway

Over the past few decades, urban freeway congestion has been highly recognized as a serious and worsening traffic problem in the world. To relieve freeway congestion, several active traffic and demand management (ATDM) methods have been developed. Among them, variable speed limit (VSL) aims at regulating freeway mainline flow upstream to meet existing capacity and to harmonize vehicle speed. However, congestion may still be inevitable even with VSL implemented due to extremely high demand in actual practice. This study modified an existing VSL strategy by adding a new local constraint to suggest an achievable speed limit during the control period. As a queue is a product of the congestion phenomenon in freeway, the incentives of a queue build-up in the applied coordinated VSL control situation were analyzed. Considering a congestion occurrence (a queue build-up) characterized by a sudden and sharp speed drop, speed contours were utilized to demonstrate the congestion distribution over a whole freeway network in various scenarios. Finally, congestion distributions found in both VSL control and non-VS control situations for various scenarios were investigated to explore the impact of the applied coordinated VSL control on the congestion distribution. An authentic stretch of Whitemud Drive (WMD), an urban freeway corridor in Edmonton, Alberta, Canada, was employed to implement this modified coordinated VSL control strategy; and a calibrated micro-simulation VISSIM model (model functions) was applied as the substitute of the real-world traffic system to test the above mentioned performance. The exploration task in this study can lay the groundwork for future research on how to improve the presented VSL control strategy for achieving the congestion mitigation effect on freeway.

Virtualized Traffic at Metropolitan Scales

Few phenomena are more ubiquitous than traffic, and few are more significant economically, socially, or environmentally. Any virtual reality system, including virtual globes and immersive multi-player worlds that are set in a modern or futuristic setting feature traffic systems. Virtual reality models can also aid in addressing the challenges of real-world traffic systems, the ever-present gridlock and congestion in cities worldwide: traffic engineers and planners can diagnose system instabilities and explore control strategies in virtual worlds reconstructed from available sensor data. To create these systems, road network models need to be created and represented. Traffic needs to be realistically and efficiently simulated. To analyze real-world scenarios, the traffic conditions need to be estimated and reconstructed. To create virtual scenarios, such as simulated cities, traffic needs to be intelligently and efficiently routed.

Variable Speed Limit Control Design for Relieving Congestion Caused by Active Bottlenecks

Variable speed limit (VSL) can be used on freeways to manage traffic flow with the goal of improving capacity. To achieve this objective, it is necessary that both speed and density dynamics be represented accurately. In this study, to deeply understand the effectiveness of VSL control, an analytical model was developed to represent drivers’ response to updated speed limits and macroscopic speed dynamical change with respect to changeable speed limits. Specifically, to model the freeway links having VSL control, the fundamental diagram (FD) was replaced with the VSL control variable in the relaxation term of the METANET. This modification led to the speed control variable appearing linearly, which is preferable for online computation. The density dynamics are based on the cell transmission model (CTM), which is introduced to estimate the transition flow among successive links with some practical constraints. It also offers flexibility in designing active bottleneck in which there is a capacity drop once feeding flow exceeds its capacity. To exploit this benefit, a modification was introduced in the FD of the density dynamics. A VSL control strategy was proposed that explicitly considers traffic characteristics at active bottleneck and its upstream-downstream segments. It can control traffic flow into any type of active bottleneck. Then, the proposed traffic dynamics with the control strategy are implemented in a freeway corridor using the model predictive control (MPC) approach. The analysis was carried out in the calibrated microsimulation model, VISSIM, within a scenario in which shock waves were present. The microsimulation model functions as a proxy for the real-world traffic system. This study reveals that, in terms of mobility, VSL is mostly effective during congestion periods.

Phasing Sequence and Signal Spacing Based Progression Bandwidth Optimization Technique

This study provided quantitative analyses about the influence of phasing sequences and the impact of intersection spacing. A large number of signal system scenarios were randomly generated to represent the real-world traffic systems. Based on simulation result, it was concluded that Leading and Lagging left turn phasing were more likely to be involved in progression bandwidth solution of signal systems with randomly distributed spacing. And the number of signals has significant impact on bandwidth attainability. Further conclusions were drawn from the fitting curve that it is impossible to obtain any two-way bandwidth solutions when more than 16 signals were involved in the system.