Routing Games Research Articles

Markov game for CV joint adaptive routing in stochastic traffic networks: A scalable learning approach

This study proposes a learning-based approach to tackle the challenge of joint adaptive routing in stochastic traffic networks with Connected Vehicles (CVs). We introduce a Markov Routing Game (MRG) to model the adaptive routing behavior of all vehicles in such networks, thereby incorporating both competitive route choices and real-time decision-making. We establish the existence of the Nash policy (i.e., optimal joint adaptive routing policy) within the MRG that enables vehicles to adapt optimally to real-time traffic conditions online through efficient communication. To enhance scalability, we innovate with a homogeneity-based mean-field approximation method and, based on that, further develop the Homogeneity-based Mean-Field Deep Reinforcement Learning (HMF-DRL) algorithm to learn the Nash policy within the MRG. Through numerical experiments on the Nguyen–Dupuis network, we demonstrate our algorithm’s ability to efficiently converge and learn the joint adaptive routing policy that significantly enhances traffic network efficiency. Furthermore, our study provides insights into the effects of travel demand, penetration of CVs, and levels of uncertainty on the performance of the joint adaptive routing policy. This paper presents a significant step towards improving network efficiency and reducing the travel time for a majority of vehicles amid uncertain traffic conditions.

Phase transitions of the price-of-anarchy function in multi-commodity routing games

We consider the behavior of the price of anarchy and equilibrium flows in nonatomic multi-commodity routing games as a function of the traffic demand. We analyze their smoothness with a special attention to specific values of the demand at which the support of the Wardrop equilibrium exhibits a phase transition with an abrupt change in the set of optimal routes. Typically, when such a phase transition occurs, the price of anarchy function has a breakpoint, i.e., is not differentiable. We prove that, if the demand varies proportionally across all commodities, then, at a breakpoint, the largest left or right derivatives of the price of anarchy and of the social cost at equilibrium, are associated with the smaller equilibrium support. This proves – under the assumption of proportional demand – a conjecture of O’Hare et al. (2016), who observed this behavior in simulations. We also provide counterexamples showing that this monotonicity of the one-sided derivatives may fail when the demand does not vary proportionally, even if it moves along a straight line not passing through the origin.

The Least Core of Routing Game Without Triangle Inequality

The Least Core of Routing Game Without Triangle Inequality

Day-to-day traffic control for networks mixed with regular human-piloted and connected autonomous vehicles

The emerging connected autonomous vehicle (CAV) technology offers a significant opportunity to address the congestion problem caused by the selfish routing behavior of human drivers. If the transportation management center (TMC) can control the routing of all vehicles on the network, they could easily reverse this effect. However, during the transition period, CAVs will share roads with human-piloted vehicles (HVs). In the era of mixed autonomy, a more realistic scenario is to route the CAVs to lead the traffic state towards the desired equilibrium. We first consider a repeated routing game in which the TMC can control the routes of CAVs to minimize the system travel time, i.e., CAVs follow the system optimal (SO) routing principle. In this manner, we can influence the travel time traversing each link, which indirectly influences HV users’ route choices. As a consequence, HV users may adjust their routes dynamically through day-to-day (DTD) learning to minimize individual travel cost. On the other hand, the SO routing strategy may induce higher travel time for CAVs than HVs within the same origin–destination (OD) pair. To address such inherent unfairness, road pricing is used to influence route choices of HVs to achieve specific supply regulation targets, e.g., traffic restraint. We present qualitative analysis on the stability of the controlled DTD dynamics and the effect of road pricing regarding the unfairness. To the best of our knowledge, this paper is one of the first to design dynamic pricing schemes for a mixed autonomy system under DTD learning dynamics. We carry out numerical experiments to demonstrate the effectiveness of the control schemes. The cooperative CAVs could reduce the total travel time when the market penetration rate of CAVs reaches the lower minimum control ratio that the network could escape the user equilibrium (UE) condition. When the market penetration rate of CAVs reaches the upper minimum control ratio, the network would achieve SO condition even though there are non-cooperative HVs. On the other hand, the pricing scheme can reduce the unfairness index and total travel time of the network when the market penetration rate of CAVs is low.

RIde-hail vehicle routing (RIVER) as a congestion game

The RIde-hail VEhicle Routing (RIVER) problem describes how drivers in a ride-hail market form a dynamic routing strategy according to the expected reward in each zone of the market. We model this decision-making problem as a Markov decision process (MDP), and view the drivers as playing an MDP routing game, with “congestion” induced by competitive matching in a zone. The meeting probability (i.e., the probability of successfully picking up a passenger after cruising in a zone for one period) is derived from a physical model, which is specified for street-hail and e-hail service modes separately, and calibrated with simulation data. We define a Wardrop equilibrium for the MDP routing game, and then prove it exists and can be obtained by solving a fixed-point problem. We further show a system optimum – corresponding to the maximum total expected reward accumulated over time by all drivers – can be achieved if drivers make individual routing decisions according to a cooperative reward rather than the personal reward. In other words, the proposed cooperative strategy “decentralizes” the system optimal solution in the MDP routing game. The results from numerical experiments, including a case study of Chicago, indicate the service mode plays a critical role in shaping system performance. While e-hail enjoys a higher fleet utilization rate than street-hail thanks to its more advanced matching technology, it may lead to a more uneven spatial distribution of vacant vehicle supply. As expected, cooperative routing improves system performance in terms of both total reward and equal distribution of supply. Yet, its benefit is much stronger in e-hail than in street-hail, especially when the supply is overly abundant. We also find, when a local demand surge occurs, the cooperative rewards rise in sync, similar to surge pricing on the supply side. Interestingly, the impact of the price surge spreads broadly in space, propagating far beyond the epicenter of the demand surge.

A unified model for the fairness mechanism‐based coordinated vehicle route guidance

AbstractCoordinated vehicle route guidance is recognized as an effective way to alleviate the Braess' paradox that new congestion is generated since numerous vehicles obey the same guidance from Google Maps. In conventional route games, decision‐makers are assumed to be completely rational (maximized utility is taken as an optimization objective). However, behavioral research finds that not only the expected profit is pursued, but also the fairness psychology is concerned by decision‐makers in game theory. In existing studies, the fairness concern of players is widely accepted in supply chains. Inspired by the fairness psychology of supply chain members, the vehicle fairness coefficient is originally proposed in this paper. Moreover, a unified fairness mechanism of the routing game is built to perfect the existing theoretical framework of route games. Specifically, the formulated vehicle fairness coefficient is incorporated into the payoff functions of a perfectly rational‐based routing game, which makes the single‐objective problem to be multiple. Compared with the current completely rational‐based route games, this improvement allows vehicles to focus not only on travel time but also on route schemes' fairness. The consideration of vehicle fairness concern leads the original symmetric payoff matrix of the route game to be asymmetric. Since the existing solving theorems are not necessarily valid for the asymmetrical multi‐player game, a matched reinforcement learning algorithm is adopted to solve this new game. An experimental study shows that the averaged fairness coefficient is increased from 0.798 to 1.00 on the basis of the equal travel time due to the working of the designed fairness mechanism.

Atomic Dynamic Routing Games with Multiple Destinations

Atomic Dynamic Routing Games with Multiple Destinations

A traveler-centric mobility game: Efficiency and stability under rationality and prospect theory.

In this paper, we study a routing and travel-mode choice problem for mobility systems with a multimodal transportation network as a "mobility game" with coupled action sets. We formulate an atomic routing game to focus on the travelers' preferences and study the impact on the efficiency of the travelers' behavioral decision-making under rationality and prospect theory. To control the innate inefficiencies, we introduce a mobility "pricing mechanism," in which we model traffic congestion using linear cost functions while also considering the waiting times at different transport hubs. We show that the travelers' selfish actions lead to a pure-strategy Nash equilibrium. We then perform a Price of Anarchy and Price of Stability analysis to establish that the mobility system's inefficiencies remain relatively low and the social welfare at a NE remains close to the social optimum as the number of travelers increases. We deviate from the standard game-theoretic analysis of decision-making by extending our mobility game to capture the subjective behavior of travelers using prospect theory. Finally, we provide a detailed discussion of implementing our proposed mobility game.

Smart navigation via strategic communications in a mixed autonomous paradigm

AbstractMotivated by the emerging mixed autonomous paradigm in cobotic order picking operations, we investigate the optimal information design to navigate human workers (HWs) who cooperate with autonomous mobile robots (AMRs) within an intralogistics system. We incorporate asymmetric information between AMRs and HWs in a routing game where connected AMRs are informed of the congestion state while HWs rely on information provided by the system. The system designs a communication policy aiming to navigate HWs away from congestion. Without strategic communications, we show that the deployment of AMRs cannot mitigate congestion unless the automation level reaches a threshold. Interestingly, we illustrate a substitution effect between automation and strategic communications when information distortion is mild. In contrast, severe information distortion complements automation due to exacerbated congestion. Furthermore, an in‐house AMR fleet is economically more efficient than a third‐party logistics service. Consequently, in‐house automation can be achieved with mild information distortion, while severe information distortion is required to complement the lack of efficiency in the third‐party AMR fleet. With simulated numerical examples to complement the analytical results, we provide managerial insights concerning the optimal information policies under different levels of automation, guiding warehouse managers in their communications with workers to achieve the best performance of the cobotic system.

Sample average approximation of conditional value-at-risk based variational inequalities

AbstractThis paper focuses on a class of variational inequalities (VIs), where the map defining the VI is given by the component-wise conditional value-at-risk (CVaR) of a random function. We focus on solving the VI using sample average approximation, where solutions of the VI are estimated with solutions of a sample average VI that uses empirical estimates of the CVaRs. We establish two properties for this scheme. First, under continuity of the random map and the uncertainty taking values in a bounded set, we prove asymptotic consistency, establishing almost sure convergence of the solution of the sample average problem to the true solution. Second, under the additional assumption of random functions being Lipschitz, we prove exponential convergence where the probability of the distance between an approximate solution and the true solution being smaller than any constant approaches unity exponentially fast. The exponential decay bound is refined for the case where random functions have a specific separable form in the decision variable and uncertainty. We adapt these results to the case of uncertain routing games and derive explicit sample guarantees for obtaining a CVaR-based Wardrop equilibria using the sample average procedure. We illustrate our theoretical findings by approximating the CVaR-based Wardrop equilibria for a modified Sioux Falls network.

LINA: A Fair Link-Grained Inter-Datacenter Traffic Scheduling Method With Deadline Guarantee

Inter-datacenter networks take the role of connecting edge clusters and datacenters distributed globally. Nowadays, inter-datacenter network topologies are becoming more and more complicated with multiple links on each path, multiple paths between two datacenters, and overlaps between different paths. Hence, a link-grained scheduling method is necessary for making routing decisions and bandwidth allocation strategies when scheduling inter-datacenter transfers. For inter-datacenter transfers, deadline guarantee and fairness are essential and conflict requirements. Unfair preemptive scheduling methods ensure deadlines but cause starvation and service interference problems, while fairly sharing bandwidth among transfers ensures fairness but causes deadlines missed. However, a link-grained scheduling method simultaneously ensuring these two conflicting objectives is still missing. In this paper, a centralized controller, called LINA, is proposed to find fair link-grained bandwidth allocation strategies, which ensure transfers can compete fairly with each other for bandwidth on links and guarantee their deadlines in the meanwhile. First, we formulate the competition among transfers as a bottleneck routing game and prove the existence and non-uniqueness of its Nash Equilibrium, which is also the optimal bandwidth allocation strategy. Then, we propose a deep reinforcement learning based algorithm to derive it. Finally, experimental results show that LINA achieves superior performance than state-of-the-art methods in terms of fairness by 50%.

A note on social learning in non-atomic routing games

This work expands on the existing body of research about Social Learning in Dynamic Non-atomic Routing Games to include infinite capacity multi-commodity instances and capacitated instances with one or more commodities. We define when a network state is identifiable, and show the network conditions sufficient to achieve almost sure learning. In capacitated instances, a sufficient condition to achieve learning is that the network respects a condition called weak capacity conservation, that is independent on the network topology. In infinite capacity instances, a sufficient condition is that the network is series-parallel in each commodity sub-network.

Competitive Comparisons of Strategic Information Provision Policies in Network Routing Games

Smart cities (i.e., cities where the sensing and communication infrastructure is developed enough to offer a central planner an informational advantage over its citizens) make it possible to envision a new kind of congestion alleviation mechanism, in which messages (whether they are they roadside signals or individual path recommendations) are actively and strategically sent to entice drivers to take socially beneficial itineraries. We compare five such information provision strategies—optimal routing, no information available, all information available, public signaling, and personal itinerary recommendation—using a competitive ratio reminiscent of the celebrated price of an anarchy metric, but adapted to stochastic networks. We also define a partial order between information provision policies by saying that a policy is maximally more efficient than another if it is more efficient on all networks and if there are instances for which the inefficiency ratio approaches the deterministic price of anarchy. With these tools, we show that a strict hierarchy exists between all policies, excluding the case of uninformed drivers. The latter is on some networks maximally better than having drivers completely informed, yet arbitrarily inefficient in other instances.

Price of anarchy for parallel link networks with generalized mean objective

The price of anarchy is the most well-known measure for quantifying the inefficiency of equilibrium flows in traffic networks and routing games. In this work, we give unifying price of anarchy bounds for atomic and non-atomic parallel link routing games with polynomial cost functions under various cost objectives including the arithmetic mean, geometric mean and worst-case cost objective. We do this through the study of the generalized p-mean as cost objective, and obtain upper bounds on the price of anarchy in terms of this parameter p. Our bounds unify existing results from the literature, and, in particular, give alternative proofs for price of anarchy results in parallel link routing games with polynomial cost functions under the geometric mean objective obtained by Vinci et al. (ACM Trans Econ Comput 10(2):41, 2022). We recover those simply as a limiting case. To the best of our knowledge, these are the first price of anarchy bounds that capture multiple cost objectives simultaneously in a closed-form expression.

Approximation and Convergence of Large Atomic Congestion Games

We consider the question of whether and in what sense, Wardrop equilibria provide a good approximation for Nash equilibria in atomic unsplittable congestion games with a large number of small players. We examine two different definitions of small players. In the first setting, we consider games in which each player’s weight is small. We prove that when the number of players goes to infinity and their weights to zero, the random flows in all (mixed) Nash equilibria for the finite games converge in distribution to the set of Wardrop equilibria of the corresponding nonatomic limit game. In the second setting, we consider an increasing number of players with a unit weight that participate in the game with a decreasingly small probability. In this case, the Nash equilibrium flows converge in total variation toward Poisson random variables whose expected values are Wardrop equilibria of a different nonatomic game with suitably defined costs. The latter can be viewed as symmetric equilibria in a Poisson game in the sense of Myerson, establishing a plausible connection between the Wardrop model for routing games and the stochastic fluctuations observed in real traffic. In both settings, we provide explicit approximation bounds, and we study the convergence of the price of anarchy. Beyond the case of congestion games, we prove a general result on the convergence of large games with random players toward Poisson games.Funding: R. Cominetti gratefully acknowledges the support of Proyecto Anillo [Grant ANID/PIA/ACT192094]. M. Scarsini’s work was partially supported by the Gruppo Nazionale per l’Analisi Matematica, la Probabilità e le loro Applicazioni, Istituto Nazionale di Alta Matematica [Grant 2020 “Random walks on random games”] and the Progetti di Rilevante Interesse Nazionale 2017 [Grant “Algorithms, Games, and Digital Markets”]. This research also received partial support from the European Cooperation in Science and Technology [Action European Network for Game Theory].

Information Design in Nonatomic Routing Games With Partial Participation: Computation and Properties

We consider a routing game among nonatomic agents where link latency functions are conditional on an uncertain state of the network. The agents have the same prior belief about the state, but only a fixed fraction receive private route recommendations or a common message, which are generated by a known randomization, referred to as <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">private</i> or <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">public signaling policy</i> , respectively. The remaining agents choose route according to Bayes Nash flow with respect to the prior. In this article, we develop a computational approach to solve the optimal information design problem, i.e., to minimize expected social latency over all public or <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">obedient</i> private signaling policies. For a fixed flow induced by nonparticipating agents, the design of an optimal private signaling policy has been shown to be a generalized problem of moments for polynomial link latency functions, and to admit an atomic solution with a provable upper bound on the number of atoms. This implies that, for polynomial link latency functions, information design can be equivalently cast as a polynomial optimization problem. This, in turn, can be arbitrarily lower bounded by a known hierarchy of semidefinite relaxations. The first level of this hierarchy is shown to be exact for the basic two link case with affine latency functions. We also identify a class of private signaling policies over which the optimal social cost is nonincreasing with an increasing fraction of participating agents for parallel networks. This is in contrast to existing results where the cost of participating agents under a <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">fixed</i> signaling policy may increase with their increasing fraction.

Routing Games in the Wild: Efficiency, Equilibration, Regret, and a Price of Anarchy Bound via Long Division

Routing games are amongst the most well studied domains of game theory. How relevant are these pen-and-paper calculations to understanding the reality of everyday traffic routing? We focus on a semantically rich dataset that captures detailed information about the daily behavior of thousands of Singaporean commuters and examine the following basic questions: - Does the traffic stabilize? - Is the system behavior consistent with latency-minimizing agents? - Is the resulting system efficient? In order to capture the efficiency of the traffic network in a way that agrees with our everyday intuition we introduce a new metric, the Free-flow Index (FFI) , which reflects the inefficiency resulting from system congestion. Along the way, we provide the first model-free computation of an upper bound to the price of anarchy utilizing only real world measurements of traffic data.

Multi-agent reinforcement learning for Markov routing games: A new modeling paradigm for dynamic traffic assignment

This paper aims to develop a paradigm that models the learning behavior of intelligent agents (including but not limited to autonomous vehicles, connected and automated vehicles, or human-driven vehicles with intelligent navigation systems where human drivers follow the navigation instructions completely) with a utility-optimizing goal and the system’s equilibrating processes in a routing game among atomic selfish agents. Such a paradigm can assist policymakers in devising optimal operational and planning countermeasures under both normal and abnormal circumstances. To this end, we develop a Markov routing game (MRG) in which each agent learns and updates her own en-route path choice policy while interacting with others in transportation networks. To efficiently solve MRG, we formulate it as multi-agent reinforcement learning (MARL) and devise a mean field multi-agent deep Q learning (MF-MA-DQL) approach that captures the competition among agents. The linkage between the classical DUE paradigm and our proposed Markov routing game (MRG) is discussed. We show that the routing behavior of intelligent agents is shown to converge to the classical notion of predictive dynamic user equilibrium (DUE) when traffic environments are simulated using dynamic loading models (DNL). In other words, the MRG depicts DUEs assuming perfect information and deterministic environments propagated by DNL models. Four examples are solved to illustrate the algorithm efficiency and consistency between DUE and the MRG equilibrium, on a simple network without and with spillback, the Ortuzar Willumsen (OW) Network, and a real-world network near Columbia University’s campus in Manhattan of New York City.

Routing Games with Edge Priorities

Routing games over time are widely studied due to various applications, e.g., transportation, road and air traffic control, logistic in production systems, communication networks like the internet, and financial flows. In this article, we present a new competitive packet routing game with edge priorities motivated by traffic and transportation. In this model a set of selfishly acting players travels through the network over time. If the number of players who want to enter an edge at the same time exceeds the inflow capacity of this edge, then edge priorities with respect to the preceding edge are used to resolve these conflicts, which is similar to right-of-way rules in traffic. We analyze the efficiency of pure Nash equilibria, present an efficient algorithm for computing equilibria in symmetric games, and show that it is NP -hard to decide whether a Nash equilibrium exists in an asymmetric game. Furthermore, we address the problem of constructing optimal priorities.

Age of information in a decentralized network of parallel queues with routing and packets losses

The paper deals with age of information (AoI) in a network of multiple sources and parallel queues with buffering capabilities, preemption in service and losses in served packets. The queues do not communicate between each other and the packets are dispatched through the queues according to a predefined probabilistic routing. By making use of the stochastic hybrid system (SHS) method, we provide a derivation of the average AoI of a system of two parallel queues (with and without buffer capabilities) and compare the results with those of a single queue. We show that known results of packets delay in Queuing Theory do not hold for the AoI. Unfortunately, the complexity of computing the average AoI using the SHS method increases highly with the number of queues. We therefore provide an upper bound of the average AoI in a system of an arbitrary number of M/M/1/(N&#x002B;1)&#x2217; queues and show its tightness in various regimes. This upper bound allows providing a tight approximation of the average AoI with a very low complexity. We then provide a game framework that allows each source to determine its best probabilistic routing decision. By using Mean Field Games, we provide an analysis of the routing game framework, propose an efficient iterative method to find the routing decision of each source and prove its convergence to the desired equilibrium.