Ramp-metering Algorithm Research Articles

A dynamic self-improving ramp metering algorithm based on multi-agent deep reinforcement learning

ABSTRACT We present a novel ramp metering algorithm that incorporates multi-agent deep reinforcement learning (DRL) techniques, which utilizes monitoring data from loop detectors. Our proposed approach employed a multi-agent DRL framework to generate optimized ramp metering schedules for each ramp meter in real-time, enhancing the operational efficiency of urban freeways with less investment. To simplify the implementation and training of the algorithm, we developed a simulation platform based on SUMO microscopic traffic simulator. We conducted a series of simulation experiments, including local and coordinated ramp metering scenarios with various traffic demands profiles. The simulation results indicate that the proposed DRL-based algorithm outperforms the state-of-the-practice ramp metering methods, considering a comprehensive evaluation index encompassing mainstream speed at the bottleneck and queue length on ramp. Additionally, the method exhibits robustness, scalability, and the potential for further improvement through online learning during implementation.

A coordinated ramp metering framework based on heterogeneous causal inference

AbstractThe coordinated ramp metering strategy aims to enhance the traffic flow on freeways by integrating the effects of multiple on‐ramps. The success of this strategy depends heavily on the metering effects of each on‐ramp and the cooperation pattern. To ensure the effectiveness of coordination control, it is essential to design the coordinated strategy based on the causal effect of each ramp flow on the freeway bottleneck. Therefore, this study investigated the heterogeneous causal effects of ramp flow on freeway bottleneck and proposed a coordinated ramp metering framework based on these causal effects. Causal inference was conducted using loop detector data. The estimated heterogeneous causal effect, which varies across traffic conditions, measures the change of bottleneck occupancy resulting from a unit increase in each on‐ramp outflow. This information quantifies the importance of each ramp in real time and can be used to update the ramp weight and metering rate of each ramp controller. To comprehensively test the proposed coordination framework, two instances with different road networks were selected. In the first instance, limiting the on‐ramp flow downstream of the bottleneck was found to be helpful in mitigating the bottleneck. Compared with isolated ramp metering, the proposed algorithm reduced travel delay by 26.0% by considering the heterogeneous causal effects of on‐ramp flow both upstream and downstream of the freeway bottleneck. In the next instance, the proposed algorithm reduced travel delay by 5.8%, compared to a well‐performed feedback‐based coordinated ramp metering algorithm. The results of this study suggest that the use of heterogeneous causal inference is effective in improving the performance of coordinated ramp metering algorithms.

Ramp Metering with Microscopic Gap Detection Algorithm Design and Empirical Acceleration Verification

Freeway on-ramp areas are susceptible to traffic congestion during peak hours. To delay or prevent the onset of congestion, ramp metering can be applied. A Ramp Metering Installation (RMI) controls the inflow from the on-ramp to the main line so that the total flow can be kept just below capacity. Current ramp metering algorithms apply macroscopic traffic characteristics, which do not entirely prevent inefficient merging behavior from occurring. This paper presents a microscopic ramp metering approach based on gap detection in the right-hand lane of the main line. As preparation for the analyses, trajectory data were collected, by which the mean and standard deviation of driver accelerations were calculated. Simulation, including driver acceleration, is used to test the ramp metering controller. Overall, it shows travel-time savings compared with no-control and compared with existing macroscopic ramp metering systems. Especially during periods of very high main line demand, the microscopic control approach is able to achieve additional travel-time savings. This way, the proposed algorithm can contribute to more efficient road usage and shorter travel times.

Corridor-Wise Eco-Friendly Cooperative Ramp Management System for Connected and Automated Vehicles

Safety, mobility, and environmental sustainability are three fundamental issues that our transportation system has been confronting for decades. Intelligent transportation systems (ITS) aim to address these problems by leveraging disruptive technologies, such as connected and automated vehicles (CAVs). The cooperative potential of CAVs enable more efficient maneuvers and operation of a group of vehicles, or even the entire traffic system. In addition, CAVs may couple with other emerging technologies such as electrification to boost overall system performance and to further mitigate the aforementioned issues. In this study, we propose a hierarchical eco-friendly cooperative ramp management system, where macroscopically, a stratified ramp metering algorithm, is deployed to coordinate all of the ramp inflow rates along a corridor according to the real-time traffic condition; microscopically, a model predictive control (MPC)-based algorithm is designed for the detailed speed control of individual CAVs. Using the shared information from CAVs, the proposed ramp management system can smooth traffic flow, improve system mobility, and decrease the energy consumption of the network. Moreover, traffic simulation has been conducted using PTV VISSIM under various congestion levels for vehicles with different powertrain types, i.e., an internal combustion engine and an electric motor. Compared to conventional ramp metering, the proposed ramp management system may improve mobility by 48.6–56.7% and save energy by 24.0–35.1%. Compared to no control scenarios, savings in travel time and energy consumption are in the ranges of 79.4–89.1% and 0.8–2.5%, respectively.

Modeling and Validating Traffic Responsive Ramp Metering in the Highway Capacity Manual Context

This paper presents a methodology for modeling traffic responsive (or adaptive) ramp metering in the freeway facilities method based on the sixth edition of the Highway Capacity Manual (HCM6). Currently, the HCM only provides an option to meter on-ramps as user input using 15-min average flow rates with a focus on planning-level analyses. As a result, the possibilities for simulating and modeling ramp meters with any traffic responsive ramp metering algorithm in the HCM context are limited. Moreover, the freeway facilities methodology in the HCM plays a vital role in the analysis of travel time reliability, which is built on a set of operational scenarios. However, with the lack of traffic responsive ramp metering, analysts are burdened with the task of manually entering average effective ramp metering rates for each on-ramp within the set of reliability scenarios. This process can require a substantial amount of time, in addition to increasing the potential for inaccuracy and bias in freeway and performance measure estimations. As a result, this paper is designed to fill a significant research gap by providing a method for analyzing traffic responsive (or adaptive) ramp metering, an active traffic and demand management strategy, using the core freeway facilities methodology in the HCM. The direct application of the method focuses on the MaxView metering algorithm. However, the proposed framework can be used to model any traffic responsive ramp metering algorithm. The results are validated using real-world sites located on the I-540 westbound freeway corridor in North Carolina.

A Novel Ramp Metering Approach Based on Machine Learning and Historical Data

The random nature of traffic conditions on freeways can cause excessive congestion and irregularities in the traffic flow. Ramp metering is a proven effective method to maintain freeway efficiency under various traffic conditions. Creating a reliable and practical ramp metering algorithm that considers both critical traffic measures and historical data is still a challenging problem. In this study we use simple machine learning approaches to develop a novel real-time ramp metering algorithm. The proposed algorithm is computationally simple and has minimal data requirements, which makes it practical for real-world applications. We conduct a simulation study to evaluate and compare the proposed approach with an existing traffic-responsive ramp metering algorithm.

Evaluating the HERO Ramp-Metering Algorithm with San Diego’s Integrated Corridor Management System Model

The FHWA project “Alternative Designs to Alleviate Freeway Bottlenecks at Merge/Diverge and Weaving Areas” aims at evaluating six different methods to mitigate merge impacts. In this paper, the following ramp-metering algorithms were tested: HEuristic Ramp metering coOrdination (HERO), Asservissement Linéaire d’entrée sur Autoroute (ALINEA), and San Diego Ramp Meter System (SDRMS). These were compared with a “do-nothing” base scenario. The algorithms were tested during a 5 h morning peak period simulation using the integrated corridor management system (ICMS) Aimsun network, a complete and wide-ranging network covering San Diego’s I-15 mainline corridor, on-off ramps and arterial roads. Whereas ALINEA is already included in the base distribution of Aimsun, a new implementation of HERO was programmed from algorithms found in the literature and adapted for use on such a network. Different measures of performance (MOP) were used to assess the efficiency of each algorithm and HERO was found to outperform all the other algorithms: gains of 1.5% over ALINEA and 4% over do-nothing on the mainline average travel time, and gains of 0.5% over ALINEA and 1.5% over do-nothing for the average weighted harmonic speed on all mainline and ramp sections. However, although tests showed that better results can be obtained on the mainline, careful calibration was needed to attain overall positive MOP and not penalize vehicles entering on ramps. The paper concludes with proposed improvements to the original algorithm.

Evaluation of Ramp Metering Impacts on Travel Time Reliability and Traffic Operations through Simulation

Ramp metering has been found to improve traffic conditions on the freeway mainline by breaking the platoons of ramp vehicles minimizing turbulence at the merge locations. The majority of the ramp metering evaluation studies have examined traffic performance under specific demand conditions, whereas travel time reliability and variability aspects have not been adequately addressed. This paper focuses on evaluating two well-known ramp metering algorithms in terms of travel time reliability as well as other performance measures such as queue lengths, throughput, and congestion duration, looking at a wide range of traffic demands throughout a calendar year. The evaluation was done through simulating an 8-mile corridor in Kansas City, KS. The results showed localized improvements due to ramp metering at the northern section of the facility, in terms of travel time reliability, throughput, and congestion duration. It was also shown that ramp metering may cause a new (possibly “hidden”) bottleneck to occur downstream, thus diluting its overall benefits when looking at an entire freeway facility. It is further noted that although ALINEA performed better than HERO on the mainline, traffic operations on the on-ramps significantly deteriorated using isolated control.

Assessing Need for Systemwide Ramp Metering Installation Warrants

Warrants for ramp metering installation have been developed by a number of U.S. states. These warrants are generally simple and are based on the traffic, geometry, and safety conditions in the immediate vicinity of each ramp (local conditions). However, advanced applications of ramp metering use system-based metering algorithms that involve metering a number of on-ramps to address system bottlenecks. These algorithms have been proved to perform better compared with local ramp metering algorithms. This situation creates a disconnect between existing agency metering warrants to install the meters and the subsequent management and operations of the ramp meters. This study focused on assessing the need for system warrants in addition to local warrants for ramp metering installation to prevent traffic breakdown at bottleneck locations. A linear programming formulation combined with consideration of the stochasticity of bottleneck capacity was used to select the ramps to be metered on the basis of the system bottlenecks. The study found that the selection of these ramps for metering—in addition to those justified by local warrants—could delay the breakdown at the system bottleneck location and improve the performance of the freeway main line. Another important benefit of selecting ramps for metering based on system operations was its distribution of on-ramp delays because of metering of more ramps and thus to a reduction in the delay experienced on the ramps selected with the existing warrants that were based on local conditions.

A Density-Based Ramp Metering Model Considering Multilane Context in Urban Expressways

As one of the most effective intelligent transportation strategies, ramp metering is regularly discussed and applied all over the world. The classic ramp metering algorithm ALINEA dominates in practical applications due to its advantages in stabilizing traffic flow at a high throughput level. Although ALINEA chooses the traffic occupancy as the optimization parameter, the classic traffic flow variables (density, traffic volume, and travel speed) may be easier obtained and understood by operators in practice. This paper presents a density-based ramp metering model for multilane context (MDB-RM) on urban expressways. The field data of traffic flow parameters is collected in Chengdu, China. A dynamic density model for multilane condition is developed. An error function represented by multilane dynamic density is introduced to adjust the different usage between lanes. By minimizing the error function, the density of mainstream traffic can stabilize at the set value, while realizing the maximum decrease of on-ramp queues. Also, VISSIM Component Object Model of Application Programming Interface is used for comparison of the MDB-RM model with a noncontrol, ALINEA, and density-based model, respectively. The simulation results indicate that the MDB-RM model is capable of achieving a comprehensive optimal result from both sides of the mainstream and on-ramp.

Implementation of Active Traffic Management Strategies for Safety on Congested Expressway Weaving Segments

In weaving segments, traffic merges, diverges, and weaves in a limited space. These traffic maneuvers might result in high crash hazards. To improve the safety of a congested expressway weaving segment, this study tested various active traffic management (ATM) strategies in microsimulations. Crash odds and the Surrogate Safety Assessment Model were used to evaluate the impact of ATM strategies on traffic safety. The crash odds were calculated based on the real-time safety analysis model for weaving segments. The strategies included ramp metering (RM), variable speed limit (VSL), and integrated RM and VSL (RM-VSL). Overall, the results showed that the ATM strategies improved the safety of the studied weaving segment. The modified ALINEA RM algorithms, which took lane occupancy and safety into consideration, outperformed the traditional ALINEA algorithm from a safety perspective. The 45 mph VSLs, which were located at the upstream of the studied weaving segment, significantly enhanced safety without notably increasing average travel time. A consolidated RM-VSL strategy was also proposed with the aim of improving traffic safety by implementing RM and VSL. In the consolidated RM-VSL strategy, the modified ALINEA RM was adjusted according to the queue length to prevent long queues on ramps. The results proved that the consolidated RM-VSL strategy reduced the number of conflicts by 16.8% and decreased the crash odds by 6.0%.

Efficiency and Equity Performance of a Coordinated Ramp Metering Algorithm

Ramp metering (RM) has been widely applied due to its effectiveness in improving motorway traffic conditions by limiting inflow from on-ramps. A great deal of experimental and simulation-based studies have proven system-wide benefits of motorways from RM. Benefits attributed to RM in the literature include reducing travel times, increasing motorway throughputs and decreasing fuel consumption and emissions. However, RM benefits might be costing more some motorway users, e.g. some on-ramp users might be experiencing longer delay than others, which leads to an unfair allocation of RM benefits. This paper presents a coordinated ramp metering strategy, which is aimed at reducing the inequity among motorway users using different on-ramps and investigates trade-offs between efficiency and equity for the proposed strategy. Total travel time is used to measure the efficiency while Gini coefficient is used to measure equity. A combined index is proposed incorporating the two measures to serve as an objective function to solve the bi-objective control design problem. The performance of the proposed strategy is verified by comparing it to a well-established coordinated ramp metering strategy HERO using micro-simulationsoftware AIMSUN. Simulation results revealed that the equity of the motorway system can be improved significantly by using the proposed strategy without compromising much on the efficiency of the system.

The Use of Cooperative Approach in Ramp Metering

To ensure higher Level of Service (LoS) at urban motorways,new traffic control concepts are being applied since in most cases there is no available space for infrastructural build-up. For urban motorways, the mostly used control methods are ramp metering combined with additional control methods like variable speed limit control (VSLC). This paper gives a review of the current ramp metering approaches with special emphasis on cooperative control concepts between ramp metering, VSLC, prohibiting lane changes system and the vehicle itself. Additionally, a learning framework for ramp metering proposed by the authors is described. The CTMSIM Matlab based macroscopic motorway simulator with ramp metering control support is used for the simulation of selected ramp metering approaches. The simulatoris also augmented to enable the development and implementation of cooperative ramp metering approaches. TheZagreb bypass is used as test case for evaluation of severaldifferent ramp metering algorithms.

Literature Review of Advancements in Adaptive Ramp Metering

Over a period spanning more than 30 years, several ramp metering algorithms have been developed to improve the operation of freeways. Many of these algorithms were deployed in several regions of the world, and field evaluations have shown their significance to improve traffic conditions on freeways and ramps. Previous reviews of ramp metering algorithms focused more on the research outcomes and evaluations of traditional metering algorithms developed in the early stage of ramp metering research. The purpose of this paper is to cover the more recent developments in ramp metering in relation to the traditional metering strategies. Several local and coordinated ramp metering algorithms were reviewed. In summary, Asservissement Linéaire d’Entrée Autoroutière (ALINEA) was found to be the most widely deployed local ramp metering strategy. The algorithm is simple and implementation costs less than other strategies. It also guarantees the targeted performance goals provided that the on-ramp has sufficient storage. Several extensions were proposed in the literature to fine-tune its performance. Among the coordinated metering strategies, zone based metering is simple to implement and easy to re-configure. System-wide adaptive ramp metering (SWARM) algorithm is more sensitive to calibrate for accurate prediction of traffic states. Heuristic ramp-metering coordination (HERO) algorithm can be useful if both local and coordinated control are desired particularly if the local control is using ALINEA. Fuzzy logic based algorithms are gaining popularity because of the simplicity and the fast re-configuration capability. Advanced real time metering system (ARMS) seems theoretically promising because of its proactive nature to prevent congestion; however, its performance is highly dependent upon accurate predictions. Finally, some guidelines were proposed for future research to develop new proposals and to extend the existing algorithms for guaranteed performance solutions.

A Local Non-restrictive Ramp Metering Strategy Based on Stochasticity of Capacity

A local non-restrictive ramp metering strategy PRO is introduced. It is based on the stochasticity of capacity. The ramp metering algorithm shows innovative features: • upstream time shifted measurements for anticipation • measurements are actuated every second • up to three vehicles per green are allowed Details of the theory of this strategy are described in the first part. At freeway B27 three ramp meters with the PRO algorithm were installed. In the second part, based on extensive detailed traffic and accident data the effects on traffic flow and safety are described. The impact is positive regarding vehicle speed, queue duration and length as well as capacity and traffic safety. The improvements of speeds, travel times and capacities are statistically significant. The ramp metering systems are highly cost effective.

New Horizontal Equity Measure for Ramp Meters

Ramp metering is a control technology used to manage the flow of traffic entering motorways and freeways, with the primary aim of minimizing congestion on the main thoroughfare. This technique has been studied and implemented globally since the 1960s. It has been shown that ramp meters improve the overall efficiency of the system; however, the distribution of the benefits and costs across users has been questioned, and this is one of the main constraints on user acceptance of the ramp metering system. The typical methodology used in the literature is to assume that the most equitable condition is when all on-ramps have the same delay across space or time. This research developed a new definition of horizontal equity for ramp meters and a proposed method for calculating it. A hypothetical microsimulation model was developed on the basis of a motorway in Sydney, Australia, and used as the platform to demonstrate how the proposed equity definition can be evaluated. To assist in the interpretation, two configurations of a ramp metering algorithm were simulated and compared. Finally, the typical equality measure used in the literature was calculated for the same scenarios and compared with the proposed equity measure. The results showed that these two measures can favor scenarios. A qualitative discussion of the expected benefits of the proposed equity measure is offered. Those expected benefits are an easy-to-communicate means of justifying the metering rates for user acceptance (rates that are arguably fairer, compared with the typical equality measure); a measure that is complementary to integration with other intelligent transportation system technology such as tolled bypass lanes; and ease of incorporation in the long-term traffic management plan.

Local ramp metering with distant downstream bottlenecks: A comparative study

The well-known feedback ramp metering algorithm ALINEA can be applied for local ramp metering or included as a key component in a coordinated ramp metering system. ALINEA uses real-time occupancy measurements from the ramp flow merging area that may be at most a few hundred meters downstream of the metered on-ramp nose. In many practical cases, however, bottlenecks with smaller capacities than the merging area may exist further downstream, which suggests using measurements from those downstream bottlenecks. Recent theoretical and simulation studies indicate that ALINEA may lead to poorly damped closed-loop behavior in this case, but PI-ALINEA, a suitable Proportional-Integral (PI) extension of ALINEA, can lead to satisfactory control performance. This paper addresses the same local ramp-metering problem in the presence of far-downstream bottlenecks, with a particular focus on the employment of PI-ALINEA to tackle three distinct cases of bottleneck that may often be encountered in practice: (1) an uphill case; (2) a lane-drop case; and (3) an un-controlled downstream on-ramp case. Extensive simulation studies are conducted on the basis of a macroscopic traffic flow model to show that ALINEA is not capable of carrying out ramp metering in these bottleneck cases, while PI-ALINEA operates satisfactorily in all cases. A field application example of PI-ALINEA is also reported with regard to a real case of far downstream bottlenecks. With its control parameters appropriately tuned beforehand, PI-ALINEA is found to be universally applicable, with little fine-tuning required for field applications.

A Multi-Hierarchical Strategy for On-Ramp Coordination

Ramp metering (RM) is an access control for motorways, in which a traffic signal is placed at on-ramps to regulate the rate of vehicles entering the motorway and thus to preserve the motorway capacity. In general, RM algorithms fall into two categories by their effective scope: local control and coordinated control. Local control algorithm determines the metering rate based on the traffic condition on adjacent motorway mainline and the on-ramp. Conversely, coordinated RM strategies make use of measurements from the entire motorway network to operate individual ramp signals for optimal performance at the network level. This study proposes a multi-hierarchical strategy for on-ramp coordination. The strategy is structured in two layers. At the higher layer, a centralised, predictive controller plans the coordination control within a long update interval based on the location of high-risk breakdown flow. At the lower layer, reactive controllers determine the metering rates of those ramps involved in the ramp coordination with a short update interval. This strategy is modelled and applied to the northbound model of the Pacific Motorway in a micro-simulation platform (AIMSUN). The simulation results show that the proposed strategy effectively delays the onset of congestion and reduces total congestion with better managed on-ramp queues.

Microscopic Estimation of Freeway Vehicle Positions From the Behavior of Connected Vehicles

Given the current connected vehicles program in the United States, as well as other similar initiatives in vehicular networking, it is highly likely that vehicles will soon wirelessly transmit status data, such as speed and position, to nearby vehicles and infrastructure. This will drastically impact the way traffic is managed, allowing for more responsive traffic signals, better traffic information, and more accurate travel time prediction. Research suggests that to begin experiencing these benefits, at least 20% of vehicles must communicate, with benefits increasing with higher penetration rates. Because of bandwidth limitations and a possible slow deployment of the technology, only a portion of vehicles on the roadway will participate initially. Fortunately, the behavior of these communicating vehicles may be used to estimate the locations of nearby noncommunicating vehicles, thereby artificially augmenting the penetration rate and producing greater benefits. We propose an algorithm to predict the locations of individual noncommunicating vehicles based on the behaviors of nearby communicating vehicles by comparing a communicating vehicle's acceleration with its expected acceleration as predicted by a car-following model. Based on analysis from field data, the algorithm is able to predict the locations of 30% of vehicles with 9-m accuracy in the same lane, with only 10% of vehicles communicating. Similar improvements were found at other initial penetration rates of less than 80%. Because the algorithm relies on vehicle interactions, estimates were accurate only during or downstream of congestion. The proposed algorithm was merged with an existing ramp metering algorithm and was able to significantly improve its performance at low connected vehicle penetration rates and maintain performance at high penetration rates.

Local Ramp Metering in the Presence of a Distant Downstream Bottleneck: Theoretical Analysis and Simulation Study

The well-known feedback ramp metering algorithm ALINEA can be applied for local ramp metering or used as a key component in a coordinated ramp metering system. ALINEA uses real-time occupancy measurements from the ramp-flow merging area that may be at most few hundred meters downstream of the metered on-ramp nose. In many practical cases, however, bottlenecks with smaller capacity than the merging area may exist further downstream for various reasons, which suggests using measurements from those further downstream bottlenecks rather than from the merging area. This paper addresses the local ramp metering problem in such a downstream bottleneck case. Theoretical analysis indicates that ALINEA may lead to a poorly damped closed-loop behavior in this case, but PI-ALINEA, which is a suitable proportional-integral (PI) extension of ALINEA, can lead to satisfactory control performance. The stability of the closed-loop ramp metering system with PI-ALINEA is rigorously proved by Lyapunov stability arguments. The root locus method is also employed to analyze the linearized closed-loop system performance of ALINEA and PI-ALINEA with and without a downstream bottleneck to provide insights on both controllers' performance. Simulation studies are conducted using a macroscopic traffic flow model to demonstrate that the ramp metering performance of ALINEA indeed deteriorates in the distant downstream bottleneck case, whereas a significant improvement is obtained using PI-ALINEA. Moreover, with its control parameters appropriately tuned, PI-ALINEA is found to be universally applicable to a range of distances between the on-ramp and downstream bottlenecks. This indicates that little fine-tuning would be necessary in field applications.