Highway On-ramps Research Articles

Research on robust decision making for intelligent connected vehicle at highway on-ramp

Reinforcement learning has demonstrated its potential in the decision-making field of autonomous driving. By engaging in trial-and-error learning and interacting with the environment, we can adapt our behavioral strategies based on reward and enhance driving decisions. Nevertheless, real-world vehicle decision-making involves intricate and diverse information, and the limited state information hinders agents from making optimal decisions, which may result in catastrophic consequences like collisions. Therefore, this paper proposes a robust deep reinforcement learning method based on the Soft Actor-Critic (SAC) algorithm for highway intelligent connected vehicle ramp merging decision-making. The model represents the vehicle features of the target lane and its adjacent lanes as an environmental state space. Additionally, a hybrid action space is designed, which combines discrete lateral actions and continuous longitudinal actions. Finally, an on-ramp simulation platform is built using actual roads and SUMO to verify the feasibility of the model. Multiple sets of comparative analysis experimental results demonstrate that the proposed method outperforms others in terms of the average reward return value, robustness value, collision rate, and success rate. Moreover, the model exhibits a stable lane change success rate under various road traffic density conditions, which indicates its robustness. The code can be obtained at https://github.com/ColinFanghz/sac-on-ramp.git .

Study on mixed traffic characteristics around highway on-ramp bottleneck using a microscopic simulation model

In the early mixed traffic environment, connected and automated vehicles (CAVs) tend to follow simple control rules for the sake of traffic safety and stability. Comprehensive analysis of overall mixed traffic performance and vehicle to vehicle interactions around highway on-ramp bottleneck is a basis for further traffic management and control. And yet CAVs in existing researches are mainly in the form of cooperative vehicular platoon, and mixed traffic scenarios within simulation software are not applicable for in-depth investigation. The purpose of this paper is to develop a microscopic simulation model to study in detail lane-based mixed traffic performance around the highway on-ramp. This model firstly considers vehicle characteristics of individual human vehicles (HVs) and CAVs by combining different microscopic traffic models. After the hypothetical on-ramp bottleneck structure is constructed, various simulation scenarios are developed under varying traffic volumes and changing market penetration rates (MPRs) of CAVs. Simulation results show that mixed traffic flow is improved in terms of congestion patterns and traffic capacity. For oscillating and nearly homogeneous congested traffic, congestion degrees are greatly reduced when CAV MPR above 30 % and 50 % respectively. Mixed traffic capacity is increased by 19.3 % when a smaller desired time gap set for CAVs, while it shows no significant change in a larger value. In the HV-HV interactions, traffic conflicts are slightly influenced by CAVs. Comparatively, conflict situations in HV-CAV interactions get worse, the proportion of time to collision (TTC) greater than 2 s monotonically declining with the rise in CAV MPR. It is concluded that traffic characteristics overall show a good momentum when MPR of CAV and the desired time gap of CAVs in mixed traffic are appropriate.

Research on Dual-Clutch Intelligent Vehicle Infrastructure Cooperative Control Based on System Delay Prediction of Two-Lane Highway On-Ramp Merging Area

Research on Dual-Clutch Intelligent Vehicle Infrastructure Cooperative Control Based on System Delay Prediction of Two-Lane Highway On-Ramp Merging Area

RAMPART: Reinforcing Autonomous Multi-agent Protection through Adversarial Resistance in Transportation

In the field of multi-agent autonomous transportation, such as automated payload delivery or highway on-ramp merging, agents routinely exchange knowledge to optimize their shared objective and adapt to environmental novelties through Cooperative Multi-Agent Reinforcement Learning (CMARL) algorithms. This knowledge exchange between agents allows these systems to operate efficiently and adapt to dynamic environments. However, this cooperative learning process is susceptible to adversarial poisoning attacks, as highlighted by contemporary research. Particularly, the poisoning attacks where malicious agents inject deceptive information camouflaged within the differential noise, a pivotal element for differential privacy (DP)-based CMARL algorithms, pose formidable challenges to identify and overcome. The consequences of not addressing this issue are far-reaching, potentially jeopardizing safety-critical operations and the integrity of data privacy in these applications. Existing research has strived to develop anomaly detection-based defense models to counteract conventional poisoning methods. Nonetheless, the recurring necessity for model offloading and retraining with labeled anomalous data undermines their practicality, considering the inherently dynamic nature of the safety-critical autonomous transportation applications. Further, it is imperative to maintain data privacy, ensure high performance, and adapt to environmental changes. Motivated by these challenges, this paper introduces a novel defense mechanism against stealthy adversarial poisoning attacks in the autonomous transportation domain, termed Reinforcing Autonomous Multi-agent Protection through Adversarial Resistance in Transportation (RAMPART). Leveraging a GAN model at each local node, RAMPART effectively filters out malicious advice in an unsupervised manner, whilst generating synthetic samples for each state-action pair to accommodate environmental uncertainties and eliminate the need for labeled training data. Our extensive experimental analysis, conducted in a Private Payload Delivery Network (PPDN) —a common application in the autonomous multi-agent transportation domain—demonstrates that RAMPART successfully defends against a DP-exploited poisoning attack with a \(30\% \) attack ratio, achieving an F1 score of 0.852 and accuracy of \(96.3\% \) in heavy-traffic environments .

Deep Multi-Agent Reinforcement Learning for Highway On-Ramp Merging in Mixed Traffic

On-ramp merging is a challenging task for autonomous vehicles (AVs), especially in mixed traffic where AVs coexist with human-driven vehicles (HDVs). In this paper, we formulate the mixed-traffic highway on-ramp merging problem as a multi-agent reinforcement learning (MARL) problem, where the AVs (on both merge lane and through lane) collaboratively learn a policy to adapt to HDVs to maximize the traffic throughput. We develop an efficient and scalable MARL framework that can be used in dynamic traffic where the communication topology could be time-varying. Parameter sharing and local rewards are exploited to foster inter-agent cooperation while achieving great scalability. An action masking scheme is employed to improve learning efficiency by filtering out invalid/unsafe actions at each step. In addition, a novel priority-based safety supervisor is developed to significantly reduce collision rate and greatly expedite the training process. A gym-like simulation environment is developed and open-sourced with three different levels of traffic densities. We exploit curriculum learning to efficiently learn harder tasks from trained models under simpler settings. Comprehensive experimental results show the proposed MARL framework consistently outperforms several state-of-the-art benchmarks.

Toward personalized decision making for autonomous vehicles: A constrained multi-objective reinforcement learning technique

Reinforcement learning promises to provide a state-of-the-art solution to the decision making problem of autonomous driving. Nonetheless, numerous real-world decision making problems involve balancing multiple conflicting or competing objectives. In addition, passengers may typically prefer to explore diversified driving modes through their specific preferences (i.e., relative importance of different objectives). Taking into account these demands, traditional reinforcement learning algorithms with applications in personalized self-driving vehicles remain challenging. Consequently, here we present a novel constrained multi-objective reinforcement learning technique for personalized decision making in autonomous driving, with the goal of learning a single model for Pareto optimal policies across the space of all possible user preferences. Specifically, a nonlinear constraint incorporating a user-specified preference and a vectorized action–value function is introduced to ensure both diversity in learned decision behaviors and efficient alignment between the user-specified preference and the corresponding optimal policy. Additionally, a constrained multi-objective actor–critic approach is advanced to approximate the Pareto optimal policies for any user-specified preferences while adhering to the nonlinear constraint. Finally, the proposed personalized decision making scheme for autonomous driving is assessed in a highway on-ramp merging scenario with dynamic traffic flows. The results demonstrate the effectiveness of our method by comparing it with classical and state-of-the-art baselines.

A Highway On-Ramp Control Approach Integrating Percolation Bottleneck Analysis and Vehicle Source Identification

Identifying the bottleneck segments and developing targeted traffic control strategies can facilitate the mitigation of highway traffic congestion. In this study, we proposed a new method for identifying the bottleneck segment in a large highway network based on the percolation theory. A targeted on-ramp control approach was further developed by identifying the major vehicle sources of the bottleneck segment. We found that the identified bottleneck segment played a crucial role in maintaining the functional connectivity of the highway network in terms of meeting the required level of service. The targeted on-ramp control approach can more effectively enhance the service level of the highway network.

Two Whitmans: Peripheral Monumentality in South Philadelphia

This article examines the contemporary politics of commemoration through analysis of the public life of two statues of Walt Whitman in South Philadelphia. Sited near a highway on-ramp and the entrance to a shopping plaza, the monuments to Whitman accompanied and celebrated significant urban spatial transformations in the mid- to late-twentieth century. At the same time, they commemorate battles over neighborhood identity and Whitman’s legacy. In recent years, local conflicts over Whitman’s legacy have been revived in activism to remove a statue of the poet in neighboring Camden, New Jersey. But, in South Philadelphia, debates about the future of public monuments have largely ignored both representations of Whitman. As I trace the lives of both statues, I argue that the quiet persistence of the two South PhiladelphiaWhitmans illustrates the complex political power of public artworks occupying awkward spatial and social positions.

Fault-Tolerant cooperative driving at highway on-ramps considering communication failure

Cooperative driving has great potential to improve traffic safety and efficiency and has been widely discussed in recent years. However, most existing researches only focus on the ideal communication environment and ignore the potential vehicle communication failure, which poses serious safety threats to traffic safety. To fill this research gap, we propose three designing principles for a fault-tolerant cooperative driving model considering communication failure. We let vehicles make real-time decision adjustments according to changes in communication conditions. Fault and non-fault vehicles can execute different decision models in a distributed manner. We test with on-ramps scenarios in simulations. Experiment results demonstrate the promising performance of the proposed model in achieving fault tolerance while reducing the impacts on traffic efficiency.

Enhancing Cooperation of Vehicle Merging Control in Heavy Traffic Using Communication-Based Soft Actor-Critic Algorithm

A promising way to improve efficiency in highway on-ramp regions is to control connected and automated vehicles (CAVs) to pass the merging section sequentially. The primary objective of this paper is to incorporate the reinforcement learning (RL) technique into vehicle merging control to achieve global cooperation. A communication protocol among RL agents is integrated with the Soft Actor-Critic (CSAC) algorithm. The parallel SAC agents with a parameter-sharing structure cooperate to optimize the common reward function. It enables CAVs to know the actions of each other so that they proactively negotiate to adjust speeds. We designed a parsimonious state representation containing crucial merging information to speed up the RL training. The effects of the merging control strategy were investigated with the simulation model in scenarios with different CAV penetration rates and traffic flow compositions. Results showed that the CSAC-based merging strategy generated collision-free merging trajectories with a short travel time while guaranteeing traffic safety in various traffic conditions. Four baselines (including two rule-based and two RL-based merging strategies) were applied in the same scenarios for comparison. It was found that the proposed merging strategy took the safest merging behaviors (zero traffic conflicts) and reduced the most significant travel time (56.9%).

Robust Decision Making for Autonomous Vehicles at Highway On-Ramps: A Constrained Adversarial Reinforcement Learning Approach

Robust Decision Making for Autonomous Vehicles at Highway On-Ramps: A Constrained Adversarial Reinforcement Learning Approach

A platoon-based cooperative optimal control for connected autonomous vehicles at highway on-ramps under heavy traffic

To improve traffic efficiency at highway on-ramps under heavy traffic, this study proposes a platoon-based cooperative optimal control algorithm for connected autonomous vehicles (CAVs). The proposed algorithm classifies CAVs on both mainline and on-ramp into multiple local platoons (LPs) according to their initial conditions (i.e., spacing and speed), which enables the algorithm to adapt to time-varying traffic volume. A distributed cooperative control for multiple LPs is designed which projects on-ramp LPs onto mainline to transform the complex 2-D multi-platoon cooperation problem into a 1-D platoon following control problem. An optimal control is applied to further consider the strict nonlinear safety spacing constraint and state limitations (e.g., maximum speed and acceleration), and an analytical solution to the optimal control is derived based on Pontryagin’s maximum principle. The consensus of intra-platoon and inter-platoon are analyzed, and sufficient conditions of the consensus are mathematically deducted based on Lyapunov stability theorem. Numerical simulations are conducted for different traffic demand levels and demand splits to verify the effectiveness of the proposed algorithm. The sensitivity analysis of maximum platoon sizes for mainline and on-ramp LPs is performed. A comparison with a baseline virtual platooning merging strategy is conducted, and results show that the proposed algorithm could significantly improve the average travel speed and traffic efficiency, and reduce total travel time.

Safety-Critical and Flexible Cooperative On-Ramp Merging Control of Connected and Automated Vehicles in Mixed Traffic

Cooperative on-ramp merging control for connected and automated vehicles (CAVs) can effectively improve traffic throughput and vehicle fuel efficiency at highway on-ramp merging bottlenecks. However, in the mixed traffic scenario where CAVs and human-driven vehicles (HDVs) coexist, the uncertain maneuvers of human drivers pose a major challenge to merging control in terms of safety and flexibility. To this end, this paper proposes a hierarchical cooperative on-ramp merging control strategy for CAVs to optimize flexible trajectories with safety guarantees in mixed traffic. First, the on-ramp merging control problem for CAVs is considered in the case of a three-vehicle coordination, resulting in an optimal control problem (OCP) coordinating on-ramp and main-lane CAVs for efficient operation while satisfying multiple safety-critical constraints. Second, a two-level hierarchical control architecture is developed to solve the OCP with mixed state-control constraints. The upper-level planner solves an unconstrained OCP with Pontryagin’s Minimum Principle to calculate an expected merging position, which is embedded in the variable time headway of safe merging constraints in the lower-level controller. Then, the controller converts the nonlinear OCP with safety-critical constraints to a quadratic programing (QP) problem by exploiting Control Barrier Functions (CBFs) and Control Lyapunov Functions (CLFs). By solving the QP efficiently, the time and energy efficient trajectory for each CAV is obtained. In addition, a receding horizon control framework is employed, which enables CAVs to determine flexible merging opportunity and tackle the disturbances caused by HDVs. Finally, comprehensive simulation results show that the proposed cooperative on-ramp merging strategy has potential in enabling merging flexibility, improving traffic efficiency and energy economy in real time.

Multi-agent Decision-Making Framework Based on Value Decomposition for Connected Automated Vehicles at Highway On-Ramps

Multi-agent Decision-Making Framework Based on Value Decomposition for Connected Automated Vehicles at Highway On-Ramps

A Game-Theory-Based Approach to Modeling Lane-Changing Interactions on Highway On-Ramps: Considering the Bounded Rationality of Drivers

In highway on-ramp sections, the conflictual interactions between a subject vehicle (merging vehicle) in the acceleration lane and a following vehicle (lagging vehicle) in the adjacent mainline can lead to traffic congestion, go–stop oscillations, and serious safety hazards. Human drivers combine their previous lane-changing experience and their perception of surrounding traffic conditions to decide whether to merge. However, the decisions that they make are not always optimal in specific traffic scenarios due to fuzzy perception and misjudgment. That is, they make lane-changing decisions in a bounded rational way. In this paper, a game-theory-based approach is used to model the interactive behavior of mandatory lane-changing in a highway on-ramp section. The model comprehensively considers vehicle interactions and the bounded rationality of drivers by modeling lane-changing behavior on on-ramps as a two-person non-zero-sum non-cooperative game with incomplete information. In addition, the Logit QRE is used to explain the bounded rationality of drivers. In order to estimate the parameters, a bi-level programming framework is built. Vehicle trajectory data from NGSIM and an unmanned aerial vehicle survey were used for model calibration and validation. The validation results were rigorously evaluated by using various performance indicators, such as the mean absolute error, root mean square error, detection rate, and false-alarm rate. It can be seen that the proposed game theory-based model was able to effectively predict merging and yielding interactions with a high degree of accuracy.

A constrained optimal control framework for vehicle platoons with delayed communication

<abstract><p>Vehicle platooning using connected and automated vehicles (CAVs) has attracted considerable attention. In this paper, we address the problem of optimal coordination of CAV platoons at a highway on-ramp merging scenario. We present a single-level constrained optimal control framework that optimizes the fuel economy and travel time of the platoons while satisfying the state, control, and safety constraints. We also explore the effect of delayed communication among the CAV platoons and propose a robust coordination framework to enforce lateral and rear-end collision avoidance constraints in the presence of bounded delays. We provide a closed-form analytical solution to the optimal control problem with safety guarantees that can be implemented in real time. Finally, we validate the effectiveness of the proposed control framework using a high-fidelity commercial simulation environment.</p></abstract>

Game Theoretic Merging Behavior Control for Autonomous Vehicle at Highway On-Ramp

In the most of previous studies, social interactions between vehicles are not considered explicitly when designing decision making and motion planning module. Nevertheless, the strong interaction exists in the most of driving scenarios, especially in highway junction. This paper presents a game theoretic merging behavior control system for autonomous vehicle at on-ramp junction considering the interaction between the merging vehicle and following vehicle in the main lane. In this work, a driving style estimation is proposed to deal with the heterogeneity of driving style. Then we adopt a model predictive control (MPC) method to plan the optimal merging trajectory based on the game theoretic decision making result. Finally, simulation and Human-in-the-loop (HIL) experiment result shows the effectiveness of our approach in on-ramp merging scenario.

Evaluating Cybersecurity Risks of Cooperative Ramp Merging in Mixed Traffic Environments

Connected and automated vehicle (CAV) technology has the potential to greatly improve transportation mobility, safety, and energy efficiency. However, ubiquitous vehicular connectivity also opens up the door to cyberattacks. In this study, we investigate cybersecurity risks of a representative cooperative traffic management application, i.e., highway on-ramp merging, in a mixed traffic environment. We develop threat models with two trajectory spoofing strategies on CAVs to create traffic congestion and devise an attack-resilient strategy for system defense. Furthermore, we leverage VEhicular NeTwork Open Simulator, a Veins extension simulator made for CAV applications, to evaluate cybersecurity risks of the attacks and performance of the proposed defense strategy. A comprehensive case study is conducted across different traffic congestion levels, penetration rates of CAVs, and attack ratios. As expected, the results show that mobility performance decreases up to 55.19% in the worst case when the attack ratio increases, as do safety and energy. With our proposed mitigation defense algorithm, the system’s cyberattack resiliency is greatly improved.

Uncovering Interpretable Internal States of Merging Tasks at Highway on-Ramps for Autonomous Driving Decision-Making

Humans make daily routine decisions based on their internal states in intricate interaction scenarios. This article presents a probabilistically reconstructive learning approach to identify the internal states of multivehicle sequential interactions when merging at highway on-ramps. We treated the merging task’s sequential decision as a dynamic, stochastic process and then integrated the internal states into a hidden Markov model (HMM)-Gaussian mixture regression (GMR) model, a probabilistic combination of an extended GMR and HMM. We also developed a variant of the expectation–maximization (EM) algorithm to estimate the model parameters and verified it based on a real-world dataset. Experiment results reveal that three interpretable internal states can semantically describe the interactive merge procedure at highway on-ramps. This finding provides a basis for developing an efficient model-based decision-making algorithm for autonomous vehicles (AVs) in a partially observable environment. Note to Practitioners—Model-based learning approaches have obtained increasing attention in decision-making design due to their stability and interpretability. This article was built upon two facts: 1) intelligent agents can only receive partially observable environmental information directly through their equipped sensors in the real world and 2) humans mainly utilize the internal states and associated dynamics inferred from observations to make proper decisions in complex environments. Similarly, autonomous vehicles (AVs) need to understand, infer, anticipate, and exploit the internal states of dynamic environments. Applying probabilistic decision-making models to AVs requires updating the internal states’ beliefs and associated dynamics after getting new observations. The designed and verified emission model in hidden Markov model (HMM)-Gaussian mixture regression (GMR) provides a modifiable functional module for online updates of the associated internal states. Experiment results based on the real-world driving dataset demonstrate that the internal states extracted using HMM-GMR can represent the dynamic decision-making process semantically and make an accurate prediction.

A Cooperative Merging Control Method for Freeway Ramps in Connected and Autonomous Driving

The highway on-ramp merging area is one of the major sections that form traffic bottlenecks. In a connected vehicle environment, V2V and V2I technologies enable real-time exchange of information, including position, speed, and acceleration. To improve the efficiency of vehicle merging at the on-ramp, this study proposes a cooperative merging control strategy for network-connected autonomous vehicles. First, the central controller designs the merging sequence and safety space for vehicles passing through the confluence point. Then, a trajectory optimization model was constructed based on vehicle longitudinal dynamics, and the PMP algorithm was used to determine the optimal control input. Finally, all vehicles follow the optimal trajectory so that the ramp vehicles merge smoothly into the mainline. Simulations verify that the proposed algorithm performs better than FIFO, with 13.2% energy savings, 41.4% increase in average speed, and 50.4% reduction in travel time over the uncontrolled merging scenario. The method is further applied to different traffic flow conditions and the results show that it can significantly improve traffic safety and mobility, while effectively reducing vehicle energy consumption. However, the traffic operation improvement is not satisfactory under low traffic demand.