Forward-backward Stochastic Differential Equations Research Articles

Leader–follower mean field LQ games: A direct method

AbstractThis paper investigates a linear quadratic mean field game with a leader and a large number of followers. The leader first gives its strategy, and then each follower optimizes its own cost. We first solve a mean field game problem, which gives the best response of followers to the leader's strategy. After applying the followers' strategies, the leader is faced to an optimal control problem driven by a high‐dimensional forward‐backward stochastic differential equations (FBSDEs). By decoupling the high‐dimensional Hamiltonian system with mean field approximations, we construct a set of decentralized strategies for all the players, which are further shown to be an ‐Stackelberg equilibrium. ‐

Decentralized strategies for finite population linear–quadratic–Gaussian games and teams

This paper is concerned with a new class of mean-field games which involve a finite number of agents. Necessary and sufficient conditions are obtained for the existence of the decentralized open-loop Nash equilibrium in terms of non-standard forward–backward stochastic differential equations (FBSDEs). By solving the FBSDEs, we design a set of decentralized strategies by virtue of two differential Riccati equations. Instead of the ɛ-Nash equilibrium in classical mean-field games, the set of decentralized strategies is shown to be a Nash equilibrium. For the infinite-horizon problem, a simple condition is given for the solvability of the algebraic Riccati equation arising from consensus. Furthermore, the social optimal control problem is studied. Under a mild condition, the decentralized social optimal control and the corresponding social cost are given.

A sufficient condition for optimal control problem of fully coupled forward‐backward stochastic systems with jumps: A state‐constrained control approach

AbstractWe study the stochastic optimal control problem for fully coupled forward‐backward stochastic differential equations (FBSDEs) with jump diffusions. A major technical challenge of such problems arises from the dependence of the (forward) diffusion term on the backward SDE and the presence of jump diffusions. Previously, this class of problems has been solved via only the stochastic maximum principle, which guarantees only the necessary condition of optimality and requires identifying unknown parameters in the corresponding variational inequality. Our paper provides an alternative approach, which constitutes the sufficient condition for optimality. Specifically, the original fully coupled FBSDE control problem (referred to as (P)) is converted into the terminal state‐constrained forward stochastic control problem (referred to as ) that includes additional (possibly unbounded) control variables. Then is solved via the backward reachability analysis, by which the value function of is expressed as the zero‐level set of the value function for the auxiliary unconstrained (forward) control problem (referred to as ). Unlike ), is an unconstrained problem, which includes additional control variables as a consequence of the martingale representation theorem. We show that the value function for is the unique viscosity solution to the associated integro‐type Hamilton‐Jacobi‐Bellman (HJB) equation. The viscosity solution analysis presented in our paper requires a new technique due to additional control variables in the Hamiltonian maximization and the presence of the nonlocal integral operator in terms of the (singular) Lévy measure. To solve the original problem (P), we reverse our approach. Specifically, we first solve to obtain the value function using the verification theorem and the viscosity solution of the HJB equation. Then is solved by characterizing the zero‐level set of the value function of , from which the optimal solution of (P) can be constructed. To illustrate the theoretical results of this paper, applications to the linear‐quadratic problem for fully coupled FBSDEs with jumps are also presented.

Solution to Forward–Backward Stochastic Differential Equations With Random Coefficients and Application to Deterministic Optimal Control

We address forward–backward stochastic differential equations (FBSDEs) with random coefficients. Differently from previous approaches, we consider FBSDEs with coefficients that are random and correlated with forward and backward variables. In addition, expectation terms appear in the forward equation, and this makes challenging to solve coupled FBSDEs, e.g., those coming from corporations’ fiscal policy planning. Using a discretization approach, an analytic solution to the FBSDEs is given in terms of coupled stochastic Riccati-type differential equations. We also apply our results to optimal deterministic control problem for Itô stochastic systems with both random and deterministic coefficients. Our results show that the discretization approach is a powerful tool to decouple complex FBSDEs.

Mixed Social Optima and Nash Equilibrium in Linear-Quadratic-Gaussian Mean-Field System

This article investigates a class of mixed stochastic linear-quadratic-Gaussian social optimization and Nash game in the context of a large-scale system. Two types of interactive agents are involved: a major agent and a large number of weakly coupled minor agents. All minor agents are cooperative to minimize the social cost as the sum of their individual costs, whereas such social cost is conflictive to that of the major agent. Thus, the major agent and all minor agents are further competitive to reach some nonzero-sum Nash equilibrium. Applying the mean-field approximations and person-by-person optimality, we obtain auxiliary control problems for the major agent and minor agents, respectively. The decentralized social strategy is derived by a class of new consistency condition (CC) system, which consists of mean-field forward–backward stochastic differential equations. The well-posedness of CC system is obtained by the discounting method. The related asymptotic social optimality for minor agents and Nash equilibrium for major–minor agents are also verified.

Convergence analysis of machine learning algorithms for the numerical solution of mean field control and games: II—the finite horizon case

We propose two numerical methods for the optimal control of McKean–Vlasov dynamics in finite time horizon. Both methods are based on the introduction of a suitable loss function defined over the parameters of a neural network. This allows the use of machine learning tools, and efficient implementations of stochastic gradient descent in order to perform the optimization. In the first method, the loss function stems directly from the optimal control problem. The second method tackles a generic forward-backward stochastic differential equation system (FBSDE) of McKean–Vlasov type, and relies on suitable reformulation as a mean field control problem. To provide a guarantee on how our numerical schemes approximate the solution of the original mean field control problem, we introduce a new optimization problem, directly amenable to numerical computation, and for which we rigorously provide an error rate. Several numerical examples are provided. Both methods can easily be applied to certain problems with common noise, which is not the case with the existing technology. Furthermore, although the first approach is designed for mean field control problems, the second is more general and can also be applied to the FBSDEs arising in the theory of mean field games.

A fully quantization-based scheme for FBSDEs

We propose a quantization-based numerical scheme for a family of decoupled forward-backward stochastic differential equations. We simplify the scheme for the control in [1] so that our approach is fully based on recursive marginal quantization and does not involve any Monte Carlo simulation for the computation of conditional expectations. We analyse in detail the numerical error of our scheme and provide some examples of application to financial mathematics.

Infinite horizon Stackelberg differential games with random coefficients under control input constraint

In this paper, we study Stackelberg stochastic differential game driven by stochastic differential equation (SDE for short) with random coefficients on infinite horizon under convex control constraint. The control enters into the drift and diffusion terms simultaneously. By employing the first-order adjoint equation (backward stochastic differential equation, BSDE for short), we are able to announce Pontryagin's maximum principle for the leader's global Stackelberg solution, within adapted open-loop structure, where the term global means that the leader's domination over the entire game duration. Due to the follower's adjoint equation develops into a BSDE, the leader will be confronted with a control problem where the state equation is a kind of infinite horizon fully coupled forward-backward stochastic differential equation (FBSDE for short). We shed light on an important application in a class of infinite horizon linear-quadratic (LQ for short) Stackelberg games in which the control process of leader's is constrained in a closed convex subset Γ 1 of full space R m 1 . Whenever the control domain is postulated to be full space, we derive a new kind of infinite horizon backward stochastic Riccati equation (BSRE for short) with asymmetric coefficients.

A class of non-zero-sum stochastic differential games between two mean–variance insurers under stochastic volatility

This paper studies the open-loop equilibrium strategies for a class of non-zero-sum reinsurance–investment stochastic differential games between two insurers with a state-dependent mean expectation in the incomplete market. Both insurers are able to purchase proportional reinsurance contracts and invest their wealth in a risk-free asset and a risky asset whose price is modeled by a general stochastic volatility model. The surplus processes of two insurers are driven by two standard Brownian motions. The objective for each insurer is to find the equilibrium investment and reinsurance strategies to balance the expected return and variance of relative terminal wealth. Incorporating the forward backward stochastic differential equations (FBSDEs), we derive the sufficient conditions and obtain the general solutions of equilibrium controls for two insurers. Furthermore, we apply our theoretical results to two special stochastic volatility models (Hull–White model and Heston model). Numerical examples are also provided to illustrate our results.

Reflected Forward-Backward Stochastic Differential Equations Driven by G-Brownian Motion with Continuous Monotone Coefficients

This paper is devoted to investigating the existence of solution to a class of reflected forward-backward stochastic differential equations driven by G-Brownian motion (G-RFBSDEs). We construct a solution to the equations by monotone convergence argument when the generator of backward equation and the drift of forward equation satisfy the monotonicity and uniformly continuous condition.

Newton Method for Stochastic Control Problems

We develop a new iterative method based on Pontryagin principle to solve stochastic control problems. This method is nothing else than the Newton method extended to the framework of stochastic controls, where the state dynamics is given by an ODE with stochastic coefficients. Each iteration of the method is made of two ingredients: computing the Newton direction, and finding an adapted step length. The Newton direction is obtained by solving an affine-linear Forward-Backward Stochastic Differential Equation (FBSDE) with random coefficients. This is done in the setting of a general filtration. We prove that solving such an FBSDE reduces to solving a Riccati Backward Stochastic Differential Equation (BSDE) and an affine-linear BSDE, as expected in the framework of linear FBSDEs or Linear-Quadratic stochastic control problems. We then establish convergence results for this Newton method. In particular, sufficient regularity of the second-order derivative of the cost functional is required to obtain (local) quadratic convergence. A restriction to the space of essentially bounded stochastic processes is needed to obtain such regularity. To choose an appropriate step length while fitting our choice of space of processes, an adapted backtracking line-search method is developed. We then prove global convergence of the Newton method with the proposed line-search procedure, which occurs at a quadratic rate after finitely many iterations. An implementation with regression techniques to solve BSDEs arising in the computation of the Newton step is developed. We apply it to the control problem of a large number of batteries providing ancillary services to an electricity network.

Linear quadratic mean field social control with common noise: A directly decoupling method

This paper is concerned with mean field linear quadratic social control with common noise, where the weight matrices of individual costs are indefinite We first obtain a set of forward–backward stochastic differential equations (FBSDEs) from variational analysis, and then construct a centralized feedback representation by decoupling the FBSDEs. By using solutions of two Riccati equations, we design a set of decentralized control laws, which is further shown to be asymptotically social optimal. The necessary and sufficient conditions are given for uniform stabilization of the systems by exploiting the relation between population state average and aggregate effect. An explicit expression of the optimal social cost is given in terms of two Riccati equations. Besides, the decentralized optimal solution is provided for mean field social control problem with finite agents.

An Equilibrium Model for the Cross Section of Liquidity Premia

We study a risk-sharing economy where an arbitrary number of heterogeneous agents trades an arbitrary number of risky assets subject to quadratic transaction costs. For linear state dynamics, the forward–backward stochastic differential equations characterizing equilibrium asset prices and trading strategies in this context reduce to a coupled system of matrix-valued Riccati equations. We prove the existence of a unique global solution and provide explicit asymptotic expansions that allow us to approximate the corresponding equilibrium for small transaction costs. These tractable approximation formulas make it feasible to calibrate the model to time series of prices and trading volume, and to study the cross section of liquidity premia earned by assets with higher and lower trading costs. This is illustrated by an empirical case study. Funding: J. Muhle-Karbe is partially supported by the CFM-Imperial Institute of Quantitative Finance. C. Yang is supported by the Hong Kong Research Grants Council [Grant 24207621] and a University Startup Grant from the Chinese University of Hong Kong.

Solving Stochastic Optimal Control Problem via Stochastic Maximum Principle with Deep Learning Method

In this paper, we aim to solve the high dimensional stochastic optimal control problem from the view of the stochastic maximum principle via deep learning. By introducing the extended Hamiltonian system which is essentially a Forward Backward Stochastic Differential Equation (FBSDE) with a maximum condition, we reformulate the original control problem as a new one. According to whether the optimal control has an explicit representation, three algorithms are proposed to solve the new control problem. Numerical results for different examples demonstrate the effectiveness of our proposed algorithms, especially in high dimensional cases. And even if the optimal control \(\tilde{u}\) in the maximum condition may not be solved explicitly, our algorithms can still deal with the stochastic optimal control problem. An important application of our proposed method is to calculate the sub-linear expectations, which correspond to a kind of fully nonlinear Partial Differential Equations (PDEs).

A Fourier Interpolation Method for Numerical Solution of FBSDEs: Global Convergence, Stability, and Higher Order Discretizations

The convolution method for the numerical solution of forward-backward stochastic differential equations (FBSDEs) was originally formulated using Euler time discretizations and a uniform space grid. In this paper, we utilize a tree-like spatial discretization that approximates the BSDE on the tree, so that no spatial interpolation procedure is necessary. In addition to suppressing extrapolation error, leading to a globally convergent numerical solution for the FBSDE, we provide explicit convergence rates. On this alternative grid the conditional expectations involved in the time discretization of the BSDE are computed using Fourier analysis and the fast Fourier transform (FFT) algorithm. The method is then extended to higher-order time discretizations of FBSDEs. Numerical results demonstrating convergence are presented using a commodity price model, incorporating seasonality, and forward prices.

Explicit Solution to the FBSDEs With Asymmetric Information and Application

This brief is focused on the linear forward-backward stochastic differential equations with asymmetric information and multiplicative noises. The main contribution is that we give the explicit solution in terms of Riccati equations. The key lies in establishing the relationship among the forward process and the backward processes. As an application, we solved a kind of stochastic LQ control problems under the asymmetric information by using the derived explicit solution.

Deep neural networks based temporal-difference methods for high-dimensional parabolic partial differential equations

Solving high-dimensional partial differential equations (PDEs) is a long-standing challenge, for which classical numerical methods suffer from the well-known curse of dimensionality. In this paper, we propose deep neural networks (NN) based temporal-difference (TD) learning methods for numerically solving high-dimensional parabolic PDEs. To this end, we approximate the solution of the original PDE with an NN function. To calculate this neural network function, we first transform the deterministic parabolic PDE to a forward-backward stochastic differential equations (FBSDE) system with the nonlinear Feynman-Kac formula, and then transform the forward updating process of FBSDE as a Markov reward process (MRP). On this basis, we approximate the solution of the PDE by training an NN function with a reinforcement learning technique described as below: we first discretize the temporal interval to a finite number of time steps and then at each time step, we generate many trajectories and design the loss function as the mean square of temporal difference error on all trajectories. With this loss function, we update the parameters of the NN function with the stochastic gradient descent (SGD) method. Numerical experiments show that comparing to some other existed deep learning methods, our method not only accelerates the computational speed but also improves the computational accuracy. In particular, the relative errors of our algorithm achieve the order of O(10−4) even the dimension of the problem is as high as 100.

Modelling Multiperiod Carbon Markets Using Singular Forward-Backward SDEs

We introduce a model for the evolution of emissions and the price of emissions allowances in a carbon market, such as the European Union Emissions Trading System (EU ETS). The model accounts for multiple trading periods, or phases, with multiple times at which compliance can occur. At the end of each trading period, the participating firms must surrender allowances for their emissions made during that period, and additional allowances can be used for compliance in the following periods. We show that the multiperiod allowance pricing problem is well-posed for various mechanisms (such as banking, borrowing, and withdrawal of allowances) linking the trading periods. The results are based on the analysis of a forward-backward stochastic differential equation with coupled forward and backward components, a discontinuous terminal condition, and a forward component that is degenerate. We also introduce an infinite-period model for a carbon market with a sequence of compliance times and with no end date. We show that, under appropriate conditions, the value function for the multiperiod pricing problem converges, as the number of periods increases, to a value function for this infinite-period model and that such functions are unique. Finally, we present a numerical example that demonstrates empirically the convergence of the multiperiod pricing problem. Funding: The Engineering and Physical Sciences Research Council and Climate-KIC funded all parts of H. Chotai’s PhD programme in the centre for doctoral training and provided many opportunities for travel to conferences and events during the programme. D. Crisan received financial support from École Polytechnique and Université Paris Diderot through visitor grants.

Mean Field Games with Common Noises and Conditional Distribution Dependent FBSDEs

In this paper, we consider the mean field game with a common noise and allow the state coefficients to vary with the conditional distribution in a nonlinear way. We assume that the cost function satisfies a convexity and a weak monotonicity property. We use the sufficient Pontryagin principle for optimality to transform the mean field control problem into existence and uniqueness of solution of conditional distribution dependent forward-backward stochastic differential equation (FBSDE). We prove the existence and uniqueness of solution of the conditional distribution dependent FBSDE when the dependence of the state on the conditional distribution is sufficiently small, or when the convexity parameter of the running cost on the control is sufficiently large. Two different methods are developed. The first method is based on a continuation of the coefficients, which is developed for FBSDE by Hu and Peng \cite{YH2}. We apply the method to conditional distribution dependent FBSDE. The second method is to show the existence result on a small time interval by Banach fixed point theorem and then extend the local solution to the whole time interval.

Robust Stackelberg Differential Game With Model Uncertainty

This article formalizes two types of modeling uncertainties in a stochastic Stackelberg linear-quadratic (LQ) differential game and then discusses the associated robust Stackelberg strategy design for either the leader or follower. Both uncertainties are primarily motivated by practical applications in engineering and management. The first uncertainty is connected to a disturbance unknown to the follower but known to the leader. A <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">soft-constraint</i> min-max control is applied by the follower to determine the optimal response, and then an <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">augmented</i> LQ forward-backward stochastic differential equation control is solved by the leader to ensure a robust strategy design. The second uncertainty involves a disturbance, the realization of which can be completely observed by the follower, but only its distribution can be accessed by the leader. Thus, a <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">hard-constraint</i> min-max control on an <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">affine-equality-constraint</i> is studied by the leader to address the <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">exact-optimal</i> robust design. Moreover, based on a weak convergence technique, a minimizing sequence of <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">near-optimal</i> robust designs is constructed, which is more tractable in computation. Some numerical results of the abovementioned robust strategies are also presented.