Mean Field Games Research Articles

On monotonicity conditions for mean field games

In this paper we propose two new monotonicity conditions that could serve as sufficient conditions for uniqueness of Nash equilibria in mean field games. In this study we aim for unconditional uniqueness that is independent of the length of the time horizon, the regularity of the starting distribution of the agents, or the regularization effect of a non-degenerate idiosyncratic noise. Through a rich class of simple examples we show that these new conditions are not only in dichotomy with each other, but also with the two widely studied monotonicity conditions in the literature, the Lasry–Lions monotonicity and displacement monotonicity conditions.

Model-free policy iteration approach to NCE-based strategy design for linear quadratic Gaussian games

This paper presents a novel completely model-free method to solve continuous-time linear quadratic (LQ) mean field games (MFGs). We consider the setting where agents can only access their own states and are coupled via their individual infinite-horizon discounted costs. For such games, a set of Nash certainty equivalence-based strategies, determined by two gain matrices and the aggregate population effect, i.e. mean field (MF) state, are developed, providing an ɛ-Nash equilibrium under certain assumptions. Note that the gain matrices are solved from two algebraic Riccati equations (AREs), respectively, and the MF state is given by the linear ordinary differential equation (ODE). All the solutions of these three equations require the exact model of system dynamics. To implement the NCE-based strategy with completely unknown system dynamics, the proposed approach develops a single-agent policy iteration (PI) and a single-agent MF generator to solve the above parameters. Specifically, first, model-based iterative equations with guaranteed convergence are developed to approximate two gain matrices. Then, the requirement of dynamical model is removed from the iteration process via the integral reinforcement learning (IRL) technique. A model-free algorithm is conducted by using measured data from a selected agent as reinforcement signals over a certain time interval. Finally, based on the obtained gain matrices, the MF state is computed offline depending on samples collected from the same agent. A numerical example is given to verify the effectiveness of the proposed method.

A mean field game model for COVID-19 with human capital accumulation

In this manuscript, we study a model of human capital accumulation during the spread of disease following an agent-based approach, where agents behave maximising their intertemporal utility. We assume that the agent interaction is of mean field type, yielding a mean field game description of the problem. We discuss how the analysis of a model including both the mechanism of change of species from one epidemiological state to the other and an optimisation problem for each agent leads to an aggregate behaviour that is not easy to describe, and that sometimes exhibits structural issues. Therefore we eventually propose and study numerically a SEIRD model in which the rate of infection depends on the distribution of the population, given exogenously as the solution to the mean field game system arising as the macroscopic description of the discrete multi-agent economic model for the accumulation of human capital. Such a model arises in fact as a simplified but tractable version of the initial one.

Mean field games of energy storage devices: A finite difference analysis

AbstractThis paper considers a mean field game of energy storage devices (ESDs) in power systems, where electrovalency is affected by the storage population. The competition processes of ESDs are characterized by a pair of coupled partial differential equations (PDEs). The so‐called mean field equilibrium (MFE) is obtained by resolving the PDEs. The resulting MFE strategy is then broadcast to all local controllers in order to independently determine a control input for each device. Since PDEs are difficult to be explicitly calculated, a pair of difference schemes are designed for approximating the exact solutions of the PDEs. Then the sufficient condition of stability is analyzed. Based on the proposed difference schemes, a difference iterative scheme is designed for numerically calculating the MFE. Some simulation results show the effectiveness of the proposed iterative schemes.

A fictitious-play finite-difference method for linearly solvable mean field games

An iterative finite difference scheme for mean field games (MFGs) is proposed. The target MFGs are derived from control problems for multidimensional systems with advection terms. For such MFGs, linearization using the Cole-Hopf transformation and iterative computation using fictitious play are introduced. This leads to an implementation-friendly algorithm that iteratively solves explicit schemes. The convergence properties of the proposed scheme are mathematically proved by tracking the error of the variable through iterations. Numerical calculations show that the proposed method works stably for both one- and two-dimensional control problems.

A Discrete Weak KAM Method for First-Order Stationary Mean Field Games

We provide a discrete weak KAM method for finding solutions of a class of first-order stationary mean field games systems. Especially, such solutions have clear dynamical meaning. First, we discretize Lax–Oleinik equations by discretizing in time, and then we prove the existence of minimizing holonomic measures for mean field games. We obtain two sequences of solutions of discrete Lax–Oleinik equations and minimizing holonomic measures for mean field games and show that converges to a solution of the stationary mean field games systems. Finally, we briefly describe how to implement a discretization in the space variable also.

Inverse problems for mean field games

The theory of mean field games (MFGs) studies the limiting behaviors of large systems where the agents interact with each other in a certain symmetric way. The running and terminal costs are critical for the agents to decide the strategies. However, in practice they are often partially known or totally unknown for the agents, while the total cost is known at the end of the game. To address this challenging issue, we propose and study several inverse problems for MFGs. When the Lagrangian is a kinetic energy, we first establish unique identifiability results, showing that one can recover either the running cost or the terminal cost from knowledge of the total cost. If the running cost is limited to the time-independent class, we can further prove that one can simultaneously recover both the running and the terminal costs. Finally, we extend the results to the setup with general Lagrangians.

Charging management and pricing strategy of electric vehicle charging station based on mean field game theory

AbstractAs an intermediary between the power grid and the electric vehicles (EVs) in the charging station, the charging pile promotes the exchange of electric energy between the power grid and EV group and also brings benefits to the charging station. It is difficult to accurately analyze the detailed energy trading behavior of a large number of charging piles with the power grid and EV group. Based on the theoretical framework of mean field game (MFG), this paper considers the battery degradation and charging efficiency taking into account the charging demand of EVs, the charging control problem of energy storage charging piles is proposed to achieve the goal of minimizing the cost of the charging station. In addition to modeling the interaction between the charging station and power grid and EVs as a finite‐time dynamic game problem, optimal decentralized energy scheduling control strategies are formulated for charging piles, and by introducing the mean field term, the optimal pricing strategy for power trading between the charging station and EVs is obtained under the mean field equilibrium condition. An iterative algorithm is proposed to solve the charging control problem, and the rationality of the problem and the effectiveness of the algorithm are verified by numerical simulation.

Unique continuation for a mean field game system

For a mean field game system, we prove the unique continuation which asserts that if Cauchy data are zero on arbitrarily chosen lateral subboundary, then the solution identically vanishes.

Decentralized Scheduling and Dynamic Pricing for Edge Computing: A Mean Field Game Approach

Edge computing provides a platform facilitating edge servers to contribute to computation offloading while economizing their resources. Traditional offloading solutions are mostly centralized, which are unscalable for large-scale edge computing networks due to complex interactions among many edge servers. Meanwhile, dynamic pricing for an operator is equally, if not more, important to accommodate users’ time-varying demands for computing services. In this paper, we develop a decentralized online optimization framework to jointly minimize the server’s cost of workload scheduling while maximizing the operator’s utility of service pricing. Specifically, we employ the mean field game to model the collective scheduling behavior of all edge servers, thereby enabling optimal decision making only based on the server’s local information. Considering the service price in practice is not adjusted as frequently as the scheduling process, we establish a two-timescale optimization framework, where workload scheduling at a small timescale is tightly embedded into service pricing at a large timescale. Using mean field approximation, we derive the closed-form expression for the minimum scheduling cost, and the approximation error is <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$O\left(\frac{1}{\sqrt{M}}\right)$</tex-math> </inline-formula> which declines as the number of edge servers <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$M$</tex-math> </inline-formula> increases. By characterizing the influence of workload scheduling on dynamic pricing, we transform the complex service utility maximization into a succinct but equivalent problem, and thus we can make use of Lyapunov optimization to determine the optimal price over time. Extensive evaluations validate the effectiveness and optimality of our scheduling and pricing schemes.

A Mean Field Games Model for Cryptocurrency Mining

We propose a mean field game model to study the question of how centralization of reward and computational power occur in Bitcoin-like cryptocurrencies. Miners compete against each other for mining rewards by increasing their computational power. This leads to a novel mean field game of jump intensity control, which we solve explicitly for miners maximizing exponential utility and handle numerically in the case of miners with power utilities. We show that the heterogeneity of their initial wealth distribution leads to greater imbalance of the reward distribution, and increased wealth heterogeneity over time, or a “rich get richer” effect. This concentration phenomenon is aggravated by a higher Bitcoin mining reward and reduced by competition. Additionally, an advantaged miner with cost advantages such as access to cheaper electricity, contributes a significant amount of computational power in equilibrium, unaffected by competition from less efficient miners. Hence, cost efficiency can also result in the type of centralization seen among miners of cryptocurrencies. This paper was accepted by Kay Giesecke, finance. Funding: A. M. Reppen is partly supported by the Swiss National Science Foundation [Grant SNF 181815]. Supplemental Material: The data files are available at https://doi.org/10.1287/mnsc.2023.4798 .

Route Choice-Based Socio-Technical Macroscopic Traffic Model

Human route choice is undeniably one of the key contributing factors towards traffic dynamics. However, most existing macroscopic traffic models are typically concerned with driving behavior and do not incorporate human route choice behavior models in their formulation. In this paper, we propose a socio-technical macroscopic traffic model that characterizes the traffic states using human route choice attributes. Essentially, such model provides a framework for capturing the Cyber-Physical-Social coupling in smart transportation systems. To derive this model, we first use Cumulative Prospect Theory (CPT) to model the human passengers’ route choice under bounded rationality. These choices are assumed to be influenced by traffic alerts and other incomplete traffic information. Next, we assume that the vehicles are operating under a non-cooperative cruise control scenario. Accordingly, human route choice segregates the traffic into multiple classes where each class corresponds to a specific route between an origin-destination pair. Thereafter, we derive a Mean Field Game (MFG) limit of this non-cooperative game to obtain a macroscopic model which embeds the human route choice attribute. Finally, we analyze the mathematical characteristics of the proposed model, present simulation studies to illustrate the model behavior and show comparison with existing traffic model.

Partially Centralized Model-Predictive Mean Field Games for controlling multi-agent systems

Fast, high-performance control of large Multi-Agent Systems (MASs) is a major challenge. We address this problem using Mean Field Games (MFGs), which deduce the macroscopic dynamics of the density distribution of the agent population from the microscopic dynamics of each agent. To control MASs using the MFG, two main problems need to be solved: preventing control performance degradation caused by distribution estimation errors and ensuring the scalability of communication. To overcome these issues, we develop a new control method called the Partially Centralized Model-Predictive MFG (PCMP-MFG). The proposed method solves the first issue by repeating distribution estimation and model-predictive control at each time; it solves the second by introducing broadcast control. Our numerical results show that the proposed PCMP-MFG method outperforms the conventional MFG-based method in a wide parameter range.

Optimal portfolio with relative performance and partial information: A mean‐field game approach

AbstractWe use a mean‐field game (MFG) formulation to investigate a family of portfolio management problems with relative performance in a partially observable financial market. The stock price process is characterized by a factor model and investors can only observe past stock price data. All investors share a common investment period and have constant absolute risk aversion (CARA) utility functions. We construct explicit linear feedback equilibria for both a finite population game and a corresponding MFG. Filtering theory and dynamic programming method are involved to solve two kinds of optimization problems with partial information. Our results show that the limiting strategy for ‐agent game can be derived as equilibrium of suitable MFG in a partially observable financial market.

Polarization and Coherence in Mean Field Games Driven by Private and Social Utility

We study a mean field game in continuous time over a finite horizon, T, where the state of each agent is binary and where players base their strategic decisions on two, possibly competing, factors: the willingness to align with the majority (conformism) and the aspiration of sticking with the own type (stubbornness). We also consider a quadratic cost related to the rate at which a change in the state happens: changing opinion may be a costly operation. Depending on the parameters of the model, the game may have more than one Nash equilibrium, even though the corresponding N-player game does not. Moreover, it exhibits a very rich phase diagram, where polarized/unpolarized, coherent/incoherent equilibria may coexist, except for T small, where the equilibrium is always unique. We fully describe such phase diagram in closed form and provide a detailed numerical analysis of the N-player counterpart of the mean field game. In this finite dimensional setting, the equilibrium selected by the population of players is always coherent (favoring the subpopulation whose type is aligned with the initial condition), but it does not necessarily minimize the cost functional. Rather, it seems that, among the coherent ones, the equilibrium prevailing is the one that most benefits the underdog subpopulation forced to change opinion.

Propagation of Chaos of Forward–Backward Stochastic Differential Equations with Graphon Interactions

In this paper, we study graphon mean field games using a system of forward–backward stochastic differential equations. We establish the existence and uniqueness of solutions under two different assumptions and prove the stability with respect to the interacting graphons which are necessary to show propagation of chaos results. As an application of propagation of chaos, we prove the convergence of n-player game Nash equilibrium for a general model, which is new in the theory of graphon mean field games.

Optimal control of the Fokker-Planck equation under state constraints in the Wasserstein space

We analyze a problem of optimal control of the Fokker-Planck equation with state constraints in the Wasserstein space of probability measures. We give first-order necessary conditions for optimality in the form of a mean-field game system of partial differential equations associated with an exclusion condition. Under suitable geometric conditions on the constraint we prove that optimal controls are Lipschitz continuous.

Machine Learning Architectures for Price Formation Models

Here, we study machine learning (ML) architectures to solve a mean-field games (MFGs) system arising in price formation models. We formulate a training process that relies on a min–max characterization of the optimal control and price variables. Our main theoretical contribution is the development of a posteriori estimates as a tool to evaluate the convergence of the training process. We illustrate our results with numerical experiments for linear dynamics and both quadratic and non-quadratic models.

The mean field games system: Carleman estimates, Lipschitz stability and uniqueness

Abstract An overdetermination is introduced in an initial condition for the second order mean field games system (MFGS). This makes the resulting problem close to the classical ill-posed Cauchy problems for PDEs. Indeed, in such a problem an overdetermination in boundary conditions usually takes place. A Lipschitz stability estimate is obtained. This estimate implies uniqueness. A new Carleman estimate is derived. This latter estimate is called “quasi-Carleman estimate”, since it contains two test functions rather than a single one in conventional Carleman estimates. These two estimates play the key role. Carleman estimates were not applied to the MFGS prior to the recent work of Klibanov and Averboukh in [M. V. Klibanov and Y. Averboukh, Lipschitz stability estimate and uniqueness in the retrospective analysis for the mean field games system via two Carleman estimates, preprint 2023, https://arxiv.org/abs/2302.10709].

Time Efficient Offloading Optimization in Automotive Multi-Access Edge Computing Networks Using Mean-Field Games

Emerging connected vehicular services, such as intelligent driving and high-definition (HD) map, are gaining increasing interest with the fast development of multi-access edge computing (MEC). For most time-sensitive and computation-intensive vehicular services, the data offloading process significantly influences the capacity and performance of MEC, especially when the number of connected vehicles is enormous. In this work, we consider data offloading optimization for a large-scale automotive MEC network. The problem is challenging due to the large number of connected vehicles and the complicated interaction between vehicles and edge servers. To tackle the scalability problem, we reformulate the original offloading optimization problem into a Mean-Field-Game (MFG) problem by abstracting the interaction among the connected vehicles as a distribution over their state spaces of task sizes, known as the mean-field term. To solve the problem efficiently, we propose a G-prox Primal-Dual-Hybrid-Gradient (PDHG) algorithm that transforms the MFG problem into a saddle-point problem. Based on our developed MFG model and G-prox PDHG algorithm, we propose the first data offloading scheme whose computation time is independent of the number of connected vehicles in automotive MEC systems. Extensive evaluation results corroborate the superior performance of our proposed scheme compared with the state-of-the-art methods.