Mean-field Game Framework Research Articles

A mean field game approach to equilibrium consumption under external habit formation

This paper studies the equilibrium consumption under external habit formation in a large population of agents. We first formulate problems under two types of conventional habit formation preferences, namely linear and multiplicative external habit formation, in a mean field game framework. In a log-normal market model with the asset specialization, we characterize one mean field equilibrium in analytical form in each problem, allowing us to understand some quantitative properties of the equilibrium strategy and conclude some financial implications caused by consumption habits from a mean-field perspective. In each problem with n agents, we construct an approximate Nash equilibrium for the n-player game using the obtained mean field equilibrium when n is sufficiently large. The explicit convergence order in each problem can also be obtained.

LQG Graphon Mean Field Games: Analysis via Graphon-Invariant Subspaces

This paper studies approximate solutions to large-scale linear quadratic stochastic games with homogeneous nodal dynamics parameters and heterogeneous network couplings within the graphon mean field game framework in [2]-[4]. A graphon time-varying dynamical system model is first formulated to study the finite and then limit problems of linear quadratic Gaussian graphon mean field games (LQG-GMFG). The Nash equilibrium of the limit problem is then characterized by two coupled graphon time-varying dynamical systems. Sufficient conditions are established for the existence of a unique solution to the limit LQG-GMFG problem. For the computation of LQG-GMFG solutions, two methods are established and employed where one is based on fixed point iterations and the other on a decoupling operator Riccati equation; furthermore, two corresponding sets of solutions are established based on spectral decompositions. Finally, a set of numerical simulations on networks associated with different types of graphons are presented.

Hybrid control for optimal visiting problems for a single player and for a crowd

In an optimal visiting problem, we want to control a trajectory that has to pass as close as possible to a collection of target points or regions. We introduce a hybrid control-based approach for the classic problem where the trajectory can switch between a group of discrete states related to the targets of the problem. The model is subsequently adapted to a mean-field game framework to study viability and crowd fluxes to model a multitude of indistinguishable players.

Price formation and optimal trading in intraday electricity markets

We develop a tractable equilibrium model for price formation in intraday electricity markets in the presence of intermittent renewable generation. Using stochastic control theory, we identify the optimal strategies of agents with market impact and exhibit the Nash equilibrium in closed form for a finite number of agents as well as in the asymptotic framework of mean field games. Our model reproduces the empirical features of intraday market prices, such as increasing price volatility at the approach of the delivery date and the correlation between price and renewable infeed forecasts, and relates these features with market characteristics like liquidity, number of agents, and imbalance penalty.

Origin-to-destination network flow with path preferences and velocity controls: A mean field game-like approach

<p style='text-indent:20px;'>In this paper we consider a mean field approach to modeling the agents flow over a transportation network. In particular, beside a standard framework of mean field games, with controlled dynamics by the agents and costs mass-distribution dependent, we also consider a path preferences dynamics obtained as a generalization of the so-called noisy best response dynamics. We introduce this last dynamics to model the fact that the agents choose their path on the basis of both the network congestion state and the observation of the agents' decision that have preceded them. We prove the existence of a mean field equilibrium obtained as a fixed point of a map over a suitable set of time-varying mass-distributions, defined edge by edge in the network. We also address the case where the admissible set of controls is suitably bounded depending on the mass-distribution on the edge itself.</p>

Price Formation and Optimal Trading in Intraday Electricity Markets with a Major Player

We study price formation in intraday electricity markets in the presence of intermittent renewable generation. We consider the setting where a major producer may interact strategically with a large number of small producers. Using stochastic control theory, we identify the optimal strategies of agents with market impact and exhibit the Nash equilibrium in a closed form in the asymptotic framework of mean field games with a major player.

Downlink Transmit Power Control in Ultra-Dense UAV Network Based on Mean Field Game and Deep Reinforcement Learning

As an emerging technology in 5G, ultra-dense unmanned aerial vehicles (UAVs) network can significantly improve the system capacity and networks coverage. However, it is still a challenge to reduce interference and improve energy efficiency (EE) of UAVs. In this paper, we investigate a downlink power control problem to maximize the EE in an ultra-dense UAV network. Firstly, the power control problem is formulated as a discrete mean field game (MFG) to imitate the interactions among a large number of UAVs, and then the MFG framework is transformed into a Markov decision process (MDP) to obtain the equilibrium solution of the MFG due to the dense deployment of UAVs. Specifically, a deep reinforcement learning-based MFG (DRL-MFG) algorithm is proposed to suppress the interference and maximize the EE by using deep neural networks (DNN) to explore the optimal power strategy for UAVs. The numerical results show that the UAVs can effectively interact with the environment to obtain the optimal power control strategy. Compared with the benchmarks algorithms, the DRL-MFG algorithm converges faster to the solution of MFG and improves the EE of UAVs. Moreover, the impact of the transmit power on EE under the different heights of the UAVs is also analyzed.

Communication-Efficient Massive UAV Online Path Control: Federated Learning Meets Mean-Field Game Theory

This paper investigates the control of a massive population of UAVs such as drones. The straightforward method of control of UAVs by considering the interactions among them to make a flock requires a huge inter-UAV communication which is impossible to implement in real-time applications. One method of control is to apply the mean field game (MFG) framework which substantially reduces communications among the UAVs. However, to realize this framework, powerful processors are required to obtain the control laws at different UAVs. This requirement limits the usage of the MFG framework for real-time applications such as massive UAV control. Thus, a function approximator based on neural networks (NN) is utilized to approximate the solutions of Hamilton-Jacobi-Bellman (HJB) and Fokker-Planck-Kolmogorov (FPK) equations. Nevertheless, using an approximate solution can violate the conditions for convergence of the MFG framework. Therefore, the federated learning (FL) approach which can share the model parameters of NNs at drones, is proposed with NN based MFG to satisfy the required conditions. The stability analysis of the NN based MFG approach is presented and the performance of the proposed FL-MFG is elaborated by the simulations.

Price Formation and Optimal Trading in Intraday Electricity Markets

We develop a tractable equilibrium model for price formation in intraday electricity markets in the presence of intermittent renewable generation. Using stochastic control theory we identify the optimal strategies of agents with market impact and exhibit the Nash equilibrium in closed form for a finite number of agents as well as in the asymptotic framework of mean field games. Our model reproduces the empirical features of intraday market prices, such as increasing price volatility at the approach of the delivery date and the correlation between price and renewable infeed forecasts, and relates these features with market characteristics like liquidity, number of agents, and imbalance penalty.

EFFORT EXPENDITURE FOR CASH FLOW IN A MEAN-FIELD EQUILIBRIUM

We study a mean-field game framework in which agents expend costly effort in order to transition into a state where they receive cash flows. As more agents transition into the cash flow receiving state, the magnitude of all remaining cash flows decreases, introducing an element of competition whereby agents are rewarded for transitioning earlier. An equilibrium is reached if the optimal expenditure of effort produces a transition intensity which is equal to the flow rate at which the continuous population enters the receiving state. We give closed-form expressions which yield equilibrium when the cash flow horizon is infinite or exponentially distributed. When the cash flow horizon is finite, we implement an algorithm which yields equilibrium if it converges. We show that in some cases, a higher cost of effort results in the agents placing greater value on the potential cash flows in equilibrium. We also present cases where the algorithm fails to converge to an equilibrium.

Mean-Field Game Strategies for Optimal Execution

ABSTRACTAlgorithmic trading strategies for execution often focus on the individual agent who is liquidating/acquiring shares. When generalized to multiple agents, the resulting stochastic game is notoriously difficult to solve in closed-form. Here, we circumvent the difficulties by investigating a mean-field game framework containing (i) a major agent who is liquidating a large number of shares, (ii) a number of minor agents (high-frequency traders (HFTs)) who detect and trade against the liquidator, and (iii) noise traders who buy and sell for exogenous reasons. Our setup accounts for permanent price impact stemming from all trader types inducing an interaction between major and minor agents. Both optimizing agents trade against noise traders as well as one another. This stochastic dynamic game contains couplings in the price and trade dynamics, and we use a mean-field game approach to solve the problem. We obtain a set of decentralized feedback trading strategies for the major and minor agents, and express the solution explicitly in terms of a deterministic fixed point problem. For a finite population of HFTs, the set of major-minor agent mean-field game strategies is shown to have a -Nash equilibrium property where as .

Effort Expenditure for Cash Flow in a Mean-Field Equilibrium

We study a mean-field game framework in which agents expend costly efforts in order to transition into a state where they receive cash flows. As more agents transition into the cash flow receiving state, the magnitude of all remaining cash flows decreases, introducing an element of competition whereby agents are rewarded for transitioning earlier. An equilibrium is reached if the optimal expenditure of effort produces a transition intensity which is equal to the flow rate at which the continuous population enters the receiving state. We give closed-form expressions which yield equilibrium when the cash flow horizon is infinite or exponentially distributed. When the cash flow horizon is finite we implement an algorithm which yields equilibrium if it converges. We show that in some cases a higher cost of effort results in the agents placing greater value on the potential cash flows in equilibrium. We also present cases where algorithm fails to converge to an equilibrium.

A mean-field game model for homogeneous flocking.

Empirically derived continuum models of collective behavior among large populations of dynamic agents are a subject of intense study in several fields, including biology, engineering, and finance. We formulate and study a mean-field game model whose behavior mimics an empirically derived nonlocal homogeneous flocking model for agents with gradient self-propulsion dynamics. The mean-field game framework provides a non-cooperative optimal control description of the behavior of a population of agents in a distributed setting. In this description, each agent's state is driven by optimally controlled dynamics that result in a Nash equilibrium between itself and the population. The optimal control is computed by minimizing a cost that depends only on its own state and a mean-field term. The agent distribution in phase space evolves under the optimal feedback control policy. We exploit the low-rank perturbative nature of the nonlocal term in the forward-backward system of equations governing the state and control distributions and provide a closed-loop linear stability analysis demonstrating that our model exhibits bifurcations similar to those found in the empirical model. The present work is a step towards developing a set of tools for systematic analysis, and eventually design, of collective behavior of non-cooperative dynamic agents via an inverse modeling approach.

Density Flow in Dynamical Networks via Mean-Field Games

Current distributed routing control algorithms for dynamic networks model networks using the time evolution of density at network edges, while the routing control algorithm ensures edge density to converge to a Wardrop equilibrium, which was characterized by an equal traffic density on all used paths. We rearrange the density model to recast the problem within the framework of mean-field games. In doing that, we illustrate an extended state-space solution approach and we study the stochastic case where the density evolution is driven by a Brownian motion. Further, we investigate the case where the density evolution is perturbed by a bounded adversarial disturbance. For both the stochastic and the worst-case scenarios, we provide conditions for the density to converge to a pre-assigned set. Moreover, we analyze such conditions from two different perspectives, repeated games with vector payoffs and inclusion theory.

Distributed Interference and Energy-Aware Power Control for Ultra-Dense D2D Networks: A Mean Field Game

Device-to-device (D2D) communications can enhance spectrum and energy efficiency due to direct proximity communication and frequency reuse. However, such performance enhancement is limited by mutual interference and energy availability, especially when the deployment of D2D links is ultra-dense. In this paper, we present a distributed power control method for ultra-dense D2D communications underlying cellular communications. In this power control method, in addition to the remaining battery energy of the D2D transmitter, we consider the effects of both the interference caused by the generic D2D transmitter to others and the interference from all others caused to the generic D2D receiver. We formulate a mean-field game (MFG) theoretic framework with the interference mean-field approximation. We design the cost function combining both the performance of the D2D communication and cost for transmit power at the D2D transmitter. Within the MFG framework, we derive the related Hamilton-Jacobi-Bellman and Fokker-Planck-Kolmogorov equations. Then, a novel energy and interference aware power control policy is proposed, which is based on the Lax-Friedrichs scheme and the Lagrange relaxation. The numerical results are presented to demonstrate the spectrum and energy efficiency performances of our proposed approach.

Mean field games models of segregation

This paper introduces and analyzes some models in the framework of mean field games (MFGs) describing interactions between two populations motivated by the studies on urban settlements and residential choice by Thomas Schelling. For static games, a large population limit is proved. For differential games with noise, the existence of solutions is established for the systems of partial differential equations of MFG theory, in the stationary and in the evolutive case. Numerical methods are proposed with several simulations. In the examples and in the numerical results, particular emphasis is put on the phenomenon of segregation between the populations.

Risk-sensitive mean-field-type games with [formula omitted]-norm drifts

We study how risk-sensitive players act in situations where the outcome is influenced not only by the state–action profile but also by the distribution of it. In such interactive decision-making problems, the classical mean-field game framework does not apply. We depart from most of the mean-field games literature by presuming that a decision-maker may include its own-state distribution in its decision. This leads to the class of mean-field-type games. In mean-field-type situations, a single decision-maker may have a big impact on the mean-field terms for which new type of optimality equations are derived. We establish a finite dimensional stochastic maximum principle for mean-field-type games where the drift functions have a p-norm structure which weaken the classical Lipschitz and differentiability assumptions. Sufficient optimality equations are established via Dynamic Programming Principle but in infinite dimension. Using de Finetti–Hewitt–Savage theorem, we show that a propagation of chaos property with virtual particles holds for the non-linear McKean–Vlasov dynamics.

Mean-Field Game Strategies for a Major-Minor Agent Optimal Execution Problem

We introduce, for the first time, a mean-field game framework for a multiple agent optimal execution problem with continuous trading. This modeling generalizes the classical single agent optimal liquidation problem to a setting with (i) a major agent who is liquidating a large portion of shares, and (ii) a number of minor agents (high-frequency traders (HFTs)) who detect and trade along with the liquidator. As in the classical framework, all agents are exposed to temporary price impact and attempt to balance their impact against price uncertainty. Unlike most other works, we account for the permanent price impact that order-flow from all agents have on the midprice and this induces a distinct cross interaction between major and minor agents. This formulation falls into the realm of stochastic dynamic game problems with mean-field couplings in the dynamics, and we analyze the problem using a mean-field game approach. We obtain a set of decentralized feedback trading strategies for the major and minor agents, and express the solution explicitly in terms of a deterministic fixed point problem. For a finite N population of HFTs, the set of major-minor agent mean-field game strategies is shown to have an epsilon-Nash equilibrium property where epsilon→0 as N→∞.