Dynamic User Equilibrium Model Research Articles

TERL: Two-Stage Ensemble Reinforcement Learning Paradigm for Large-Scale Decentralized Decision Making in Transportation Simulation

Transportation simulation is non-trivial due to the co-existence of thousands of heterogeneous decision makers (or vehicles). Such large-scale decision making is intrinsically a complex decentralized problem, the resolution of which is at the forefront of transportation simulation. Despite many physical or mathematical models proposed to date, underlying them is usually a set of universal rules plus some random perturbations to characterize vehicular movements. Inspired by the decision-making mechanism of rational human beings (i.e., learning iteratively from experience), this study proposes a novel two-stage ensemble reinforcement learning (TERL) paradigm for large-scale decentralized decision making in transportation simulation, in order to enhance the computational efficiency and thus scalability of RL to practical applications at scale. After establishing a problem-specific Markov decision process, the first stage utilizes clustering to group heterogeneous vehicles into quasi-homogeneous clusters. A representative RL model (or agent) is employed for each cluster where the included vehicles share and jointly optimize the policy parameters. The second stage develops an ensemble control strategy based on representative RL models for vehicles isolated as noise during clustering. While vehicles are simultaneously simulated, RL models are separately trained with cluster-specific experience replay. Application of TERL to the classical multi-user dynamic route choice problem in a real-world network of the Gusu District in Suzhou, China demonstrates the effectiveness of the proposed approach in deriving desirable simulation results, compared with the classical shortest path model and the dynamic user equilibrium model.

Incentivized user-based relocation strategies for moderating supply–demand dynamics in one-way car-sharing services

One-way car-sharing services have drawn much attention in recent years in transportation research and practice. The flexible operating scheme stimulates patronage but causes unbalanced distributions of the shared cars and consequently low utilization of the services. In light of the empirical evidence that users are willing to cooperate with the operators for relocations, this paper formulates the supply–demand dynamics of shared cars under four user-based relocation strategies, including a naive one without incentive and three alternatives with incentives. We formulate the triggering conditions and travel disutilities responding to the incentive-based strategies based on the concept of load-space–time paths. To systematically assess the effects of the strategies, we propose an incentive-based boundedly rational dynamic user equilibrium (IBR-DUE) model in a multimodal transportation system. A cluster-based column generation algorithm is developed to solve the IBR-DUE model without path enumeration in a space–time supernetwork. Numerical examples demonstrate that the alternative drop-off incentive outperforms other strategies and can improve the utilization and service rates about 2 times on average.

Analysis of activity duration-related charging behavioral responses of electric vehicle travelers to charging services

Electric vehicles (EVs) have drawn much attention in recent years due to the advantage of zero on-road carbon emission. The government subsidies and rapid development of electricity charging technologies promote the roll-out of EVs. The boost of EVs brings challenges to the deployment of EV charging services. Given that travelers may need to charge their EVs while conducting out-of-home activities, this study analyzes the activity duration-related charging behavioral responses of private EV travelers to charging services in an equilibrium model. The charging services concern the operating policies related to the spatial allocation of charging opportunities and charging pricing. Considering the trade-offs between travel and activity duration-related charging choices, a boundedly rational dynamic user equilibrium (BR-DUE) model is proposed to evaluate the two operating policies. The case study of the Eindhoven network demonstrates that the operating policies, if appropriately set up, can improve the accessibility to charging services with improved charging service rate and decreased queuing time.

Charging station planning based on the accumulation prospect theory and dynamic user equilibrium

Large-scale use of electric vehicles will greatly increase the traffic pressure on urban road network. Therefore, planning of charging stations for electric vehicles considering charging demand and transportation network is particularly important for the coordinated development of electric vehicles and intelligent transportation. Under the condition of bounded rationality, this paper considers such factors as the travel utility perception difference between the users of fuel vehicles and electric vehicles, the time-varying of traffic flow, the location and service level of charging stations. On this basis, combining the cumulative prospect theory, dynamic traffic flow allocation and charging demands, a two-level programming model is established to solve the problem of charging station site selection. The upper layer is a system optimal model, the goal is to minimize the travel time of the network. The lower model describes the time-variability of departure time and the randomness of charging and travel behaviors, establishes the dynamic user equilibrium model and designs the heuristic algorithm. The validity of the model and algorithm is verified by a numerical example. Through the simulation experiment, the optimal location scheme of charging station under different electric vehicle proportion is obtained, and the driving characteristics of two types of vehicles are analyzed. Compared with the traditional model, it is found that the charging station planning considering bounded rationality can achieve higher road network traffic efficiency with fewer charging piles.

Incorporating vehicle self-relocations and traveler activity chains in a bi-level model of optimal deployment of shared autonomous vehicles

The combination of autonomous vehicles (AVs) and free-floating car-sharing scheme is expected to deliver high potentials of both through effective AV self-relocations. Little research has been done on the deployment of shared AVs (SAVs) considering the interplays among SAV relocations, supply-demand dynamics, and travelers’ multi-modal multi-activity schedules. This study aims to propose a bi-level system optimal model inclusive of a new hub-based relocation strategy to moderate the supply and demand of SAVs. The lower-level captures travelers’ activity-travel scheduling behavior by an extended dynamic user equilibrium model and the upper-level determines the hub locations, fleet size, and initial distribution of SAVs. A heuristic algorithm based on Lagrangian relaxation is developed to solve the network design problem. Numerical examples demonstrate that SAV relocations can significantly influence travelers’ daily schedules and enhance mobility efficiency in the multi-modal transport system. We also find that the proposed hub-based relocation strategy outperforms two common SAV relocation strategies in the literature.

A dynamic user equilibrium model for multi-region macroscopic fundamental diagram systems with time-varying delays

Macroscopic fundamental diagram (MFD) has been widely used for aggregate modeling of urban traffic network dynamics to tackle the dimensionality problem of microscopic approaches. This paper contributes to the state-of-the-art by proposing a dynamic user equilibrium (DUE) model that enables simultaneous route choice and departure time choice under the MFD framework for various applications such as park-and-ride, vehicle dispatching and relocation. To better capture the traffic flow propagation and to adapt to the fast time-varying demand, the state-dependent travel time function is integrated into the MFD dynamics as an endogenous time-varying delay. The multi-region MFD dynamics with saturated state and inflow constraints is then used as the network loading model to formulate the DUE model through the lens of the differential variational inequality. Necessary conditions for the DUE are analytically derived using the Pontryagin minimum principle. Difficulties raised in handling the dynamic state-dependent nonlinear travel time functions, state and inflow constraints are addressed without model linearization nor enforcing constant delay assumption as conventionally done in the literature. The additional cost induced by inflow capacity and accumulation constraints can capture the hypercongestion represented by the downward sloping part of the MFD without actually activating traffic congestion. Numerical examples solved by using time-discretization solution algorithm illustrate the DUE characteristics and the corresponding dynamic external costs induced by constraints.

Bilevel dynamic continuum model for housing allocation and transportation emission problems in an urban city

In this study, we examine housing allocation and transportation emission problems in an urban city. Housing development patterns have a significant influence on travel behavior and related transportation emissions. We consider a hypothetical city with one central business district in which the road network is assumed to be sufficiently dense to be viewed as a continuum. We establish a bilevel dynamic continuum model to describe the relationships among housing provision, the transportation system, and traffic-related emissions. In the lower-level subprogram, housing allocation is considered in a simultaneous departure time and route-choice dynamic traffic user equilibrium model, in which travelers choose routes and departure times that satisfy dynamic user-optimal principles. In the upper-level subprogram, the total emissions are minimized by optimizing the housing allocation. The finite-volume method, the projection method, the method of successive averages, and the Frank–Wolfe method are applied to solve the presented model based on unstructured meshes. A numerical experiment for an urban area with one central business district is given to demonstrate the effectiveness of the model and the numerical algorithm.

Path-Based Dynamic User Equilibrium Model with Applications to Strategic Transportation Planning

This study proposes an analytical capacity constrained dynamic traffic assignment (DTA) model along with an efficient path-based algorithm. The model can be applied to analyzing dynamic traffic demand management (TDM) strategies, but its specific feature is allowing for an evaluation of advanced traveler information systems (ATIS) within the strategic transportation planning framework. It is an extension of a former DTA model, where each link is given an infinite capacity and is assumed to be completely traversed inside any time interval it is reach by a path. Thereby, the length of time intervals should be very longer than the link travel times, and the link capacity constraints are overlooked. The paper rests on three key ideas to overcome these restrictions: (1) adding path-link fraction variables to the base model, allowing path flows to spread out over time intervals on long links; (2) uniformly dividing each link into smaller parts (segments), so that each part is more likely to be traversed inside a time interval; (3) imposing a dynamic penalty function on each link, thereby the capacity constraint can be included. The proposed DTA algorithm decomposes the augmented model in terms of origin-destination (OD) pairs and departure time intervals, and utilizes a dynamic column generation technique for generating active paths between the OD pairs. The optimal solution to a one-link network demonstrates that the model is able to approximate the dynamic flow propagation over a link with sensible accuracy. Besides, investigation of the results for a small test network reveals that the algorithm performs very well in computing temporal link flows and queuing delays. Finally, numerical experiments on a real life network indicate that the algorithm converges sufficiently fast and provides path information for each time interval. The network is further used to show the algorithm is capable of assessing a dynamic TDM strategy as well as an ATIS system.

Resilience Enhancement Strategies for Power Distribution Network Coupled With Urban Transportation System

Traffic lights play a critical role in mitigating traffic congestions, which can be energized by distributed generators (DGs) when power outages occur in urban areas. This paper studies the resilience enhancement strategy by line hardening and DG placement when outages occur in distribution lines and traffic lights in the coupled power distribution system and urban transportation system (PDS-UTS). A tri-level optimization model is formulated with a limited budget for line hardening and DG placement to minimize the cost of load shedding and aggregated vehicle travel time. The first level determines line hardening and DG placement strategies, the second level searches for the worst case of faulted lines that would maximize load shedding and aggregated vehicle travel time, and the third level minimizes the corresponding costs of load shedding and travel. In urban transportation system, a dynamic user equilibrium model is established in a cell transmission model and solved by a linear complementarity approach. As the number of unavailable lines and traffic lights are definite in the inner-most level, the coupled PDS-UTS is considered as two decoupled systems. Accordingly, the tri-level model is converted into an equivalent bi-level model through Karush–Kuhn–Tucker conditions, which is then solved by a greedy search method. Case studies corroborate the effectiveness of the proposed model and relevant solution method for the coupled PDS-UTS.

Tolerance-based strategies for extending the column generation algorithm to the bounded rational dynamic user equilibrium problem

The column generation (CG) algorithm has been widely applied to traffic assignment problems due to its capability of circumventing path enumeration. Incorporating bounded rationality (BR) and dynamics, this paper proposes four tolerance-based strategies for extending the CG algorithm to the bounded rational dynamic user equilibrium model (BR-DUE): (i) a tolerance-based minimum disutility path search strategy is developed to allow travelers seeking satisfactory paths; (ii) a self-adjusted convergence threshold strategy is applied for fast convergence at the intermediate iterations; (iii) a varied temporal resolution scheme, combining exploration and exploitation, is suggested to assign flows to narrow time regions rather than to the whole time horizon; and (iv) a path search skipping strategy is introduced by comparing the lower bound of travel disutility and the minimum disutility between the OD pairs. With these strategies, an efficient tolerance-based column generation (TBCG) algorithm for BR-DUE is developed. Numerical examples are provided to demonstrate that the TBCG algorithm leads to significant computation time reductions without the expense of solution quality, of which the speedup factors are around two compared with using the original CG algorithm.

Path-based capacity-restrained dynamic traffic assignment algorithm

ABSTRACTThis study proposes a new fast convergence path-based dynamic traffic assignment algorithm. The algorithm is based on an analytical capacity-restrained dynamic user equilibrium model, in which link travel times are temporal functions of link flows. In previous models the length of time intervals has to be much longer (about five times) than the link travel times in each time interval using the simplifying assumption that each link will be fully traversed within the time interval that a path reaches its tail node. Otherwise, these models will make large errors and will exhibit difficulties in attaining optimality. This paper tackles the problem by proposing a systematic way of link segmentation. The numerical and case study results of the proposed algorithm show its excellent performance. The algorithm rapidly converges to optimality and provides path information over time intervals, which make it suitable for evaluating the intelligent transportation systems in strategic transport planning.

An Integrated Scenario Ensemble-Based Framework for Hurricane Evacuation Modeling: Part 1-Decision Support System.

This article introduces a new integrated scenario-based evacuation (ISE) framework to support hurricane evacuation decision making. It explicitly captures the dynamics, uncertainty, and human-natural system interactions that are fundamental to the challenge of hurricane evacuation, but have not been fully captured in previous formal evacuation models. The hazard is represented with an ensemble of probabilistic scenarios, population behavior with a dynamic decision model, and traffic with a dynamic user equilibrium model. The components are integrated in a multistage stochastic programming model that minimizes risk and travel times to provide a tree of evacuation order recommendations and an evaluation of the risk and travel time performance for that solution. The ISE framework recommendations offer an advance in the state of the art because they: (1) are based on an integrated hazard assessment (designed to ultimately include inland flooding), (2) explicitly balance the sometimes competing objectives of minimizing risk and minimizing travel time, (3) offer a well-hedged solution that is robust under the range of ways the hurricane might evolve, and (4) leverage the substantial value of increasing information (or decreasing degree of uncertainty) over the course of a hurricane event. A case study for Hurricane Isabel (2003) in eastern North Carolina is presented to demonstrate how the framework is applied, the type of results it can provide, and how it compares to available methods of a single scenario deterministic analysis and a two-stage stochastic program.

A Link-Based Differential Complementarity System Formulation for Continuous-Time Dynamic User Equilibria with Queue Spillbacks

This paper proposes a link-based continuous-time dynamic user equilibrium model for networks with single destinations. The model captures realistic queue spillbacks by applying the double-queue concept at the link level and developing a new nodal model that extends the link-level dynamics to the network level. The departure-time choice, route choice, and other equilibrium conditions are introduced to complete the model. The proposed model is a differential complementary system formulation with time-varying, state-dependent delays. Approximations on travel times are constructed to simplify the model. Numerical tests are developed to illustrate the application of this model. The online appendix is available at https://doi.org/10.1287/trsc.2017.0752 .

Incorporating free-floating car-sharing into an activity-based dynamic user equilibrium model: A demand-side model

Free-floating car-sharing (FFC) has recently received increasing attention due to the flexibility in mobility services. Existing studies related to FFC mainly focus on the analysis of operational management and user preferences. Efforts to model the dynamic choices of free-floating shared cars (SCs) in individuals’ daily multi-modal multi-activity trip chains have still been rare. This study proposes a tolerance-based dynamic user equilibrium model of activity-travel scheduling that formulates free-floating SC as an alternative transport mode for conducting daily activities. The model embeds the choice of SC into daily trip chains by extending the state-of-the-art multi-state supernetwork representation. The dynamic traffic flows and supply-demand interactions of SCs are captured endogenously. Moreover, traveler heterogeneity and different pricing schemes are taken into account. A path-flow swapping method is suggested to solve the tolerance-based dynamic user equilibrium model. Numerical examples of various scenarios demonstrate that fleet size, distribution, and rental-parking price of FFC significantly influence the choice of SC and activity-travel pattern.

Dynamic Continuum Model with Elastic Demand for a Polycentric Urban City

This paper presents the development of a macroscopic dynamic traffic assignment model for continuum transportation systems with elastic demand. A reactive dynamic user equilibrium model is extended to simulate network equilibrium problems for an urban area with multiple central business districts (CBDs). Each copy of traffic flow reactively makes route choice decisions to minimize the total travel cost from origin to destination, based on instantaneous traffic information from a radio broadcasting service or route guidance system. The elastic demand function for each copy of flow is associated with its total instantaneous travel cost. The model is solved by a cell-centered finite volume method for conservation law equations and a fast sweeping method for Eikonal-type equations on unstructured grids. Numerical experiments for an urban traffic network with two CBDs are presented to verify the rationality of the model and the validity of the numerical method. The numerical results indicate that the model captures some macroscopic characteristics of two copies of flow interacting in time-varying urban transportation systems, e.g., the spatial distributions of the flow density and flux and the path choice behavior in response to elastic demand, and can describe traffic equilibrium phenomena, such as self-organization and traffic congestion build-up and dissipation.

Multiclass, simultaneous route and departure time choice dynamic traffic assignment with an embedded spatial queuing model

ABSTRACTThis paper develops a complementarity formulation for a multi-user class, simultaneous route and departure time choice dynamic user equilibrium (DUE) model. A path-based multiclass cell transmission model (mCTM) is embedded to propagate the traffic flow on the network. Heterogeneous user classes are incorporated in the new formulation and heterogeneity is based on different preferred arrival times, cost perception for travel time, early and late arrival penalties. Multiple model properties have been showed. The proposed model is solved as an equivalent non-monotone variational inequality (VI) problem defined on a product set. A modified proximal point algorithm is used to solve the proposed non-monotone VI problem. Numerical results show that the solution approach is able to find the equilibrium or close to equilibrium solutions. The new formulation and solution approach show the feasibility of solving the multiclass DUE problem for general traffic networks.

Determining the Impact of Personal Mobility Carbon Allowance Schemes in Transportation Networks

Personal mobility carbon allowance (PMCA) schemes are designed to reduce carbon consumption from transportation networks. PMCA schemes influence the travel decision process of users and accordingly impact the system metrics including travel time and greenhouse gas (GHG) emissions. We develop a multi-user class dynamic user equilibrium model to evaluate the transportation system performance when PMCA scheme is implemented. The results using Sioux-Falls test network indicate that PMCA schemes can achieve the emissions reduction goals for transportation networks. Further, users characterized by high value of travel time are found to be less sensitive to carbon budget in the context of work trips. Results also show that PMCA scheme can lead to higher emissions for a path compared with the case without PMCA because of flow redistribution. The developed network equilibrium model allows to examine the change in system states at different carbon allocation levels and to design parameters of PMCA schemes accounting for population heterogeneity.

Routing aspects of electric vehicle drivers and their effects on network performance

This study investigates the routing aspects of battery electric vehicle (BEV) drivers and their effects on the overall traffic network performance. BEVs have unique characteristics such as range limitation, long battery recharging time, and recuperation of energy lost during the deceleration phase if equipped with regenerative braking system (RBS). In addition, the energy consumption rate per unit distance traveled is lower at moderate speed than at higher speed. This raises two interesting questions: (i) whether these characteristics of BEVs will lead to different route selection compared to conventional internal combustion engine vehicles (ICEVs), and (ii) whether such route selection implications of BEVs will affect the network performance. With the increasing market penetration of BEVs, these questions are becoming more important. This study formulates a multi-class dynamic user equilibrium (MCDUE) model to determine the equilibrium flows for mixed traffic consisting of BEVs and ICEVs. A simulation-based solution procedure is proposed for the MCDUE model. In the MCDUE model, BEVs select routes to minimize the generalized cost which includes route travel time, energy related costs and range anxiety cost, and ICEVs to minimize route travel time. Results from numerical experiments illustrate that BEV drivers select routes with lower speed to conserve and recuperate battery energy while ICEV drivers select shortest travel time routes. They also illustrate that the differences in route choice behavior of BEV and ICEV drivers can synergistically lead to reduction in total travel time and the network performance towards system optimum under certain conditions.

Solving a Dynamic User Equilibrium model based on splitting rates with Gradient Projection algorithms

This article shows how Gradient Projection (GP) algorithms are capable of solving with high precision a Dynamic User Equilibrium (UE) model based on Splitting Rates, i.e. turning movements fractions by destination.Dynamic Traffic Assignment (DTA) is formulated as a Variational Inequality problem defined on temporal profiles of arc conditional probabilities that express a sequence of deterministic route choices taken at nodes by road users directed toward each destination.Congestion is represented through a macroscopic traffic model capable to reproduce a range of phenomena having increasing complexity, from links with bottleneck to intersections with spillback. Different time discretizations, from few seconds to few minutes, are also possible, which allows a range of applications from planning to operation.This assignment model, which is fully link based, is proved to be equivalent to a path based formulation. It also allows for the computation of a handy gap function for analyzing convergence to equilibrium.Numerical experiments on test networks are presented, showing that the proposed GP algorithms converge to dynamic equilibrium in a reasonable number of iterations, outperforming the Method of Successive Averages (MSA).

A cell-based multiple vehicle type dynamic user equilibrium model with physical queues

This paper proposes a cell-based multiple vehicle type dynamic user equilibrium model with physical queues. A single-type traffic flow model is extended to a general case with multiple vehicle types that can be partly solved by the time-space discretization method. Then, a network version of the multiple vehicle type cell transmission model is given. An integrated variational inequality (VI) formulation is presented to capture the complex traveler choice behaviors such as route and departure time choices. Furthermore, a genetic algorithm with a flow-swapping method is adopted to solve the VI problem. Two examples are used to evaluate the properties of this formulation. The results show that the model can reflect dynamic phenomena, such as multiple vehicle type speed consistent under congested conditions, queue formation and dissipation and so on. Moreover, the solutions can approximately follow the multiple vehicle type dynamic route and departure time user equilibrium conditions.