Mean-field Control Research Articles

Stochastic maximum principle for mean-field type singular optimal control problem with discounted cost

Stochastic maximum principle for mean-field type singular optimal control problem with discounted cost

Extended Mean Field Control Problems: Stochastic Maximum Principle and Transport Perspective

We study mean field stochastic control problems where the cost function and the state dynamics depend upon the joint distribution of the controlled state and the control process. We prove suitable ...

Cemracs 2017: numerical probabilistic approach to MFG

This project investigates numerical methods for solving fully coupled forward-backward stochastic differential equations (FBSDEs) of McKean-Vlasov type. Having numerical solvers for such mean field FBSDEs is of interest because of the potential application of these equations to optimization problems over a large population, say for instance mean field games (MFG) and optimal mean field control problems. Theory for this kind of problems has met with great success since the early works on mean field games by Lasry and Lions, see [29], and by Huang, Caines, and Malhamé, see [26]. Generally speaking, the purpose is to understand the continuum limit of optimizers or of equilibria (say in Nash sense) as the number of underlying players tends to infinity. When approached from the probabilistic viewpoint, solutions to these control problems (or games) can be described by coupled mean field FBSDEs, meaning that the coefficients depend upon the own marginal laws of the solution. In this note, we detail two methods for solving such FBSDEs which we implement and apply to five benchmark problems. The first method uses a tree structure to represent the pathwise laws of the solution, whereas the second method uses a grid discretization to represent the time marginal laws of the solutions. Both are based on a Picard scheme; importantly, we combine each of them with a generic continuation method that permits to extend the time horizon (or equivalently the coupling strength between the two equations) for which the Picard iteration converges.

Desynchronization control of FitzHugh-Nagumo networks with random topology

This paper studies the influence of the coupling type on the network desynchro-nization and proposes the algorithms to control it. We consider the FitzHugh-Nagumo network with random topology and different coupling values for different nodes. The simple mean field control which is similar for all nodes can be applicable for desynchronization of FitzHugh-Nagumo network with direct coupling. However, the network with diffusive coupling is much harder to desynchronize. In this case for large enough coupling values one should use different control algorithms for different nodes to desynchronize the network.

Mean field production output control with sticky prices: Nash and social solutions

This paper presents an application of mean field control to dynamic production optimization with sticky prices and adjustment costs. Both noncooperative and cooperative solutions are considered. By solving auxiliary limiting optimal control problems subject to consistent mean field approximations, two sets of decentralized strategies are obtained and further shown to asymptotically attain Nash equilibria and social optima, respectively. A numerical example is given to compare market prices, firms’ outputs and costs under the two solution frameworks.

Resource Allocation in Wireless Powered IoT System: A Mean Field Stackelberg Game-Based Approach.

The IoT system has become a significant component of next generation networks, and drawn a lot of research interest in academia and industry. As the sensor nodes in the IoT system are always battery-limited devices, the power control problem is a serious problem in the IoT system which needs to be solved. In this paper, we research the resource allocation in the wireless powered IoT system, which includes one hybrid access point (HAP) and many wireless sensor nodes, to obtain the optimal power level for information transmission and energy transfer simultaneously. The relationship between the HAP and the sensor nodes are formulated as the Stackelberg game, and the dynamic variations of the energy for both the HAP and IoT devices are formulated through the dynamic game with mean field control. Then the power control in the wireless powered IoT system is formulated as a mean field Stackelberg game model. We aim to minimize the transmission cost for each sensor node based on optimally power resource allocation. Meanwhile, we attempt to minimize the energy transfer cost based on power control. As a result, the optimal solutions based on the mean field control of the sensor nodes and the HAP are achieved through dynamic programming theory and the law of large numbers, and -Nash equilibriums can be obtained. The energy variations for both the sensor nodes and HAP after the control of resource allocation based on the proposed approach are verified based on the simulation results.

Позиционные стратегии в задачах управления средним полем на пространстве конечного числа состояний

Позиционные стратегии в задачах управления средним полем на пространстве конечного числа состояний

A Mean-Field Stochastic Maximum Principle for Optimal Control of Forward-Backward Stochastic Differential Equations with Jumps via Malliavin Calculus

This paper considers a mean-field type stochastic control problem where the dynamics is governed by a forward and backward stochastic differential equation (SDE) driven by Levy processes and the information available to the controller is possibly less than the overall information. All the system coefficients and the objective performance functional are allowed to be random, possibly non-Markovian. Malliavin calculus is employed to derive a maximum principle for the optimal control of such a system where the adjoint process is explicitly expressed.

Bellman equation and viscosity solutions for mean-field stochastic control problem

We consider the stochastic optimal control problem of McKean−Vlasov stochastic differential equation where the coefficients may depend upon the joint law of the state and control. By using feedback controls, we reformulate the problem into a deterministic control problem with only the marginal distribution of the process as controlled state variable, and prove that dynamic programming principle holds in its general form. Then, by relying on the notion of differentiability with respect to probability measures recently introduced by [P.L. Lions, Cours au Collège de France: Théorie des jeux à champ moyens, audio conference 2006−2012], and a special Itô formula for flows of probability measures, we derive the (dynamic programming) Bellman equation for mean-field stochastic control problem, and prove a verification theorem in our McKean−Vlasov framework. We give explicit solutions to the Bellman equation for the linear quadratic mean-field control problem, with applications to the mean-variance portfolio selection and a systemic risk model. We also consider a notion of lifted viscosity solutions for the Bellman equation, and show the viscosity property and uniqueness of the value function to the McKean−Vlasov control problem. Finally, we consider the case of McKean−Vlasov control problem with open-loop controls and discuss the associated dynamic programming equation that we compare with the case of closed-loop controls.

Robust decentralised mean field control in leader following multi‐agent systems

This study addresses a robust counterpart of the deterministic mean field control in a multi-agent system. A decentralised mean field algorithm is proposed to solve a min–max control problem for a large population of heterogeneous agents. In the proposed leader following scheme, the leader tracks a reference signal which is unknown to the followers and each follower tracks a convex combination of the population state average and the leader's state. The leader plays a robust min–max game against disturbance and the followers play a mean field e N -Nash game against each other and at the same time, each follower plays a robust min–max game against the disturbance. For all the players, finite horizon quadratic cost is considered. In the proposed decentralised algorithm, followers do not need the knowledge about each of leader's and other followers' states and they only use an estimate of the population state average. In this way, propose a policy iteration method which guarantees the convergence to the saddle point mean field e N -Nash solution. The proposed method is applied to a large population of agents and compared with centralised algorithm to demonstrate the results.

$\epsilon$-Nash Equilibria for Partially Observed LQG Mean Field Games With a Major Player

Huang (2010) and Nguyen and Huang (2012) solved the linear quadratic mean field systems and control problem in the case where there is a major agent (i.e. non-asymptotically vanishing as the population size goes to infinity) together with a population of minor agents (i.e. individually asymptotically negligible). The new feature in this case is that the mean field becomes stochastic and then, by minor agent state extension, the existence of $\epsilon$ -Nash equilibria together with the individual agents' control laws that yield the equilibria may be established. This paper presents results initially announced by Caines and Kizilkale (2013, 2014) where it is shown that if the major agent's state is partially observed by the minor agents, and if the major agent completely observes its own state, all agents can recursively generate estimates (in general individually distinct) of the major agent's state and the mean field, and thence generate feedback controls yielding $\epsilon$ -Nash equilibria.

Singular mean-field control games

ABSTRACTThis article studies singular mean field control problems and singular mean field two-players stochastic differential games. Both sufficient and necessary conditions for the optimal controls and for the Nash equilibrium are obtained. Under some assumptions the optimality conditions for singular mean-field control are reduced to a reflected Skorohod problem, whose solution is proved to exist uniquely. Motivations are given as optimal harvesting of stochastic mean-field systems, optimal irreversible investments under uncertainty and mean-field singular investment games. In particular, a simple singular mean-field investment game is studied, where the Nash equilibrium exists but is not unique.

A general characterization of the stochastic optimal combined control of mean field stochastic systems with application

In this paper, a general characterization of the optimal stochastic combined control for mean-field jump-systems is derived by applying mixed convex-spike perturbation method. The diffusion coefficient depends on the continuous control variable and the control domain is not necessary convex. In our combined mean-field control problem, we discuss two classes of jumps for the state processes, the inaccessible jumps which caused by Poisson martingale measure and the predictable ones which caused by the singularity of the control variable. Markowitz’s mean–variance portfolio selection problem with intervention control is discussed.

State Estimation for the Individual and the Population in Mean Field Control With Application to Demand Dispatch

This paper concerns state estimation problems in a mean field control setting. In a finite population model, the goal is to estimate the joint distribution of the population state and the state of a typical individual. The observation equations are a noisy measurement of the population. The general results are applied to demand dispatch for regulation of the power grid, based on randomized local control algorithms. In prior work by the authors it has been shown that local control can be carefully designed so that the aggregate of loads behaves as a controllable resource with accuracy matching or exceeding traditional sources of frequency regulation. The operational cost is nearly zero in many cases. The information exchange between grid and load is minimal, but it is assumed in the overall control architecture that the aggregate power consumption of loads is available to the grid operator. It is shown that the Kalman filter can be constructed to reduce these communication requirements,

On optimal mean-field control problem of mean-field forward-backward stochastic system with jumps under partial information

This paper considers the problem of partially observed optimal control for forward-backward stochastic systems driven by Brownian motions and an independent Poisson random measure with a feature that the cost functional is of mean-field type. When the coefficients of the system and the objective performance functionals are allowed to be random, possibly non-Markovian, Malliavin calculus is employed to derive a maximum principle for the optimal control of such a system where the adjoint process is explicitly expressed. The authors also investigate the mean-field type optimal control problem for the system driven by mean-field type forward-backward stochastic differential equations (FBSDEs in short) with jumps, where the coefficients contain not only the state process but also its expectation under partially observed information. The maximum principle is established using convex variational technique. An example is given to illustrate the obtained results.

Decentralized Convergence to Nash Equilibria in Constrained Deterministic Mean Field Control

This paper considers decentralized control and optimization methodologies for large populations of systems, consisting of several agents with different individual behaviors, constraints and interests, and affected by the aggregate behavior of the overall population. For such large-scale systems, the theory of aggregative and mean field games has been established and successfully applied in various scientific disciplines. While the existing literature addresses the case of unconstrained agents, we formulate deterministic mean field control problems in the presence of heterogeneous convex constraints for the individual agents, for instance arising from agents with linear dynamics subject to convex state and control constraints. We propose several model-free feedback iterations to compute in a decentralized fashion a mean field Nash equilibrium in the limit of infinite population size. We apply our methods to the constrained linear quadratic deterministic mean field control problem and to the constrained mean field charging control problem for large populations of plug-in electric vehicles.

Mean-field stochastic Volterra optimal singular control with Poisson jumps∗∗This work was supported by SERB project DST New Delhi India under the F. No: YSS/2014/000447 dated 20-11-2015.

Abstract This paper considers an infinite horizon mean-field type of stochastic Volterra singular control problem. The dynamical system is governed by a Ito-Levy processes and a standard one dimensional independent Brownian motion. Stochastic Volterra controlled system is difficult to manipulate by standard methods such as dynamic programming and classical maximum principle, because the presence of memory terms in the dynamics of the system. In this study, Malliavin calculus is a useful tool to overcome these difficulties. Necessary and sufficient condition for the aforementioned system is established by using Malliavin calculus, convex perturbation technique and transversality condition. Finally an example is given to show that the application of the theoretical study.

Inverse Problem for Non-Viscous Mean Field Control: Example From Traffic

This paper presents an inverse problem for mean field games where we find the mean field problem statement for which the given dynamics is the solution. We use distributed traffic as an example and derive the classic Lighthill Whitham Richards (LWR) model as a solution of the non-viscous mean field game. We also derive the same model by choosing a different problem where we use travel time, which is a distributed parameter, as the cost for the optimal control. We then study the stationary versions of these two problems and provide numerical solutions for the same.

Stochastic maximum principle for mean-field forward-backward stochastic control system with terminal state constraints

In this paper, we consider an optimal control problem with state constraints, where the control system is described by a mean-field forward-backward stochastic differential equation (MFFBSDE, for short) and the admissible control is mean-field type. Making full use of the backward stochastic differential equation theory, we transform the original control system into an equivalent backward form, i.e., the equations in the control system are all backward. In addition, Ekeland’s variational principle helps us deal with the state constraints so that we get a stochastic maximum principle which characterizes the necessary condition of the optimal control. We also study a stochastic linear quadratic control problem with state constraints.

Decentralised dynamic games for large population stochastic multi‐agent systems

In this study, the authors consider the large population dynamic games where each agent evolves according to a dynamic equation containing the input average of all agents. The long time average (LTA) cost that each agent aims to minimise is coupled with other agents’ states via a population state average (PSA), which is also known as the mean field term. In order to design decentralised controls, the Nash certainty equivalence is introduced. It is shown that the resulting decentralised mean field control laws lead the system to mean-consensus asymptotically as time goes to infinity. The stability property of the mass behaviour and the almost sure asymptotic Nash equilibrium property of the optimal controls are also guaranteed, and the case with non-linear system dynamics is also discussed. In addition, the influence of inaccurate mean field information on individual agent is analysed. Finally, they investigate the socially cooperative formulation where the objective is to minimise the social cost as the sum of all individual LTA costs containing the PSA. In this case, they show that the decentralised mean field social controls are the same as the mean field Nash controls for infinite population systems.