Dimensional Optimal Control Problem Research Articles

An optimal control problem with state constraints in a spatio-temporal economic growth model on networks

We introduce a spatial economic growth model where space is described as a network of interconnected geographic locations and we study a corresponding finite-dimensional optimal control problem on a graph with state constraints. Economic growth models on networks are motivated by the nature of spatial economic data, which naturally possess a graph-like structure: this fact makes these models well-suited for numerical implementation and calibration. The network setting is different from the one adopted in the related literature, where space is modeled as a subset of a Euclidean space, which gives rise to infinite dimensional optimal control problems. After introducing the model and the related control problem, we prove existence and uniqueness of an optimal control and a regularity result for the value function, which sets up the basis for a deeper study of the optimal strategies. Then, we focus on specific cases where it is possible to find, under suitable assumptions, an explicit solution of the control problem. Finally, we discuss the cases of networks of two and three geographic locations.

Generalized fractional-order Legendre wavelet method for two dimensional distributed order fractional optimal control problem

This paper is concerned with a two-dimensional fractional optimal control problem whose governing equations are distributed order fractional differential equations in the Caputo sense. A generalized fractional-order Legendre wavelet method has been used to solve the two-dimensional distributed-order fractional optimal control problem. An exact formula for the Riemann–Liouville integration of generalized fractional-order Legendre wavelet has been derived by using regularized beta functions. This formula and the two-dimensional Gauss–Legendre integration formula have been used to solve the two-dimensional distributed order fractional optimal control problem. Moreover, an L2-error estimate in the approximation of an unknown function with a generalized fractional-order Legendre wavelet has been derived and the estimated order has been verified for a given function. Furthermore, convergence analysis for the proposed method has been presented. In the last, two test problems have been considered to illustrate the efficiency of the proposed method.

Necessary optimality conditions for two dimensional fractional optimal control problems and error estimates for the numerical approximation

AbstractThis paper presents the necessary optimality conditions and a new method for solving a class of two dimension fractional optimal control problem (FOCP‐2D) based on shifted Legendre polynomials. By using the Lagrange multiplier method and integration by part formula within the calculus of variations, the necessary optimality conditions are derived as two‐point fractional‐order boundary value problem. Fractional operators of shifted Legendre polynomial are computed and the necessary optimality conditions have been converted into a system of algebraic equations by utilizing these fractional operators. L2‐error estimates in the approximation of a function and its fractional derivative by shifted Legendre polynomial have been derived. Moreover, convergence analysis of the proposed method has also been discussed and in the last, illustrative examples are included to demonstrate the effectiveness and applicability of the proposed method.

Solving Stochastic Optimal Control Problem via Stochastic Maximum Principle with Deep Learning Method

In this paper, we aim to solve the high dimensional stochastic optimal control problem from the view of the stochastic maximum principle via deep learning. By introducing the extended Hamiltonian system which is essentially a Forward Backward Stochastic Differential Equation (FBSDE) with a maximum condition, we reformulate the original control problem as a new one. According to whether the optimal control has an explicit representation, three algorithms are proposed to solve the new control problem. Numerical results for different examples demonstrate the effectiveness of our proposed algorithms, especially in high dimensional cases. And even if the optimal control \(\tilde{u}\) in the maximum condition may not be solved explicitly, our algorithms can still deal with the stochastic optimal control problem. An important application of our proposed method is to calculate the sub-linear expectations, which correspond to a kind of fully nonlinear Partial Differential Equations (PDEs).

A Stochastic Model of Economic Growth in Time-Space

We deal with an infinite horizon, infinite dimensional stochastic optimal control problem arising in the study of economic growth in time-space. Such a problem has been the object of various papers in deterministic cases when the possible presence of stochastic disturbances is ignored (see, e.g., [P. Brito, The Dynamics of Growth and Distribution in a Spatially Heterogeneous World, working paper 2004/14, ISEG-Lisbon School of Economics and Management, University of Lisbon, 2004], [R. Boucekkine, C. Camacho, and G. Fabbri, J. Econom. Theory, 148 (2013), pp. 2719--2736], [G. Fabbri, J. Econom. Theory, 162 (2016), pp. 114--136], and [R. Boucekkine, G. Fabbri, S. Federico, and F. Gozzi, J. Econom. Geography, 19 (2019), pp. 1287--1318]). Here we propose and solve a stochastic generalization of such models where the stochastic term, in line with the standard stochastic economic growth models (see, e.g., the books [A. G. Malliaris and W. A. Brock, Stochastic Methods in Economics and Finance, Advanced Textbooks in Economics 17, North Holland, 1982, Chapter 3] and [H. Morimoto, Stochastic Control and Mathematical Modeling: Applications in Economics, Cambridge Books, 2010, Chapter 9]), is a multiplicative one, driven by a cylindrical Wiener process. The problem is studied using the dynamic programming approach. We find an explicit solution of the associated HJB equation, use a verification type result to prove that such a solution is the value function, and find the optimal feedback strategies. Finally, we use this result to study the asymptotic behavior of the optimal trajectories.

Optimal Portfolio Choice With Path-Dependent Labor Income: Finite Retirement Time

This paper extends the project initiated in and studies a lifecycle portfolio choice problem with borrowing constraints and finite retirement time in which an agent receives labor income that adjusts to financial market shocks in a path dependent way. The novelty here, with respect to, is the fact that we have a finite retirement time, which makes the model more realistic, but harder to solve. The presence of both path-dependency, as in, and finite retirement, leads to a two-stage infinite dimensional stochastic optimal control problem, a family of problems which, to our knowledge, has not yet been treated in the literature. We solve the problem completely, and find explicitly the optimal controls in feedback form. This is possible because we are able to find an explicit solution to the associated infinite dimensional Hamilton-Jacobi-Bellman (HJB) equation, even if state constraints are present. Note that, differently from, here the HJB equation is of parabolic type, hence the work to identify the solutions and optimal feedbacks is more delicate, as it involves, in particular, time-dependent state constraints, which, as far as we know, have not yet been treated in the infinite dimensional literature. The explicit solution allows us to study the properties of optimal strategies and discuss their financial implications.

A new hybrid method for two dimensional nonlinear variable order fractional optimal control problems

AbstractThis paper presents an efficient operational matrix method for two dimensional nonlinear variable order fractional optimal control problems (2D‐NVOFOCP). These problems include the nonlinear variable order fractional dynamical systems (NVOFDS) described by partial differential equations such as the diffusion‐wave, convection‐diffusion‐wave and Klein‐Gordon equations. The variable order fractional derivative is defined in the Caputo type. The proposed hybrid method is based on the transcendental Bernstein series (TBS) and the generalized shifted Chebyshev polynomials (GSCP). The new operational matrices of derivatives are generated for the mentioned polynomials. The state and control functions are expressed by the TBS and GSCP with free coefficients and control parameters. These expansions are substituted in the performance index and the resulting operational matrices are employed to extract algebraic equations from the approximated NVOFDS. The constrained extremum is obtained by coupling the algebraic constraints from the dynamical system and the initial and boundary conditions with the algebraic equation extracted from the performance index by means of a set of unknown Lagrange multipliers. The convergence analysis is discussed and several numerical experiments illustrate the efficiency and accuracy of the proposed method.

A One Dimensional Elliptic Distributed Optimal Control Problem with Pointwise Derivative Constraints

We consider a one dimensional elliptic distributed optimal control problem with pointwise constraints on the derivative of the state. By exploiting the variational inequality satisfied by the derivative of the optimal state, we obtain higher regularity for the optimal state under appropriate assumptions on the data. We also solve the optimal control problem as a fourth order variational inequality by a C 1 finite element method, and present the error analysis together with numerical results.

On the relation between turnpike properties and dissipativity for continuous time linear quadratic optimal control problems

<p style='text-indent:20px;'>The paper is devoted to analyze the connection between turnpike phenomena and strict dissipativity properties for continuous-time finite dimensional linear quadratic optimal control problems. We characterize strict dissipativity properties of the dynamics in terms of the system matrices related to the linear quadratic problem. These characterizations then lead to new necessary conditions for the turnpike properties under consideration, and thus eventually to necessary and sufficient conditions in terms of spectral criteria and matrix inequalities. One of the key novelty of these results is the possibility to encompass the presence of state and input constraints.

Trajectory Optimization of Autonomous Agents With Spatio-Temporal Constraints

This article addresses the problem of optimally controlling trajectories of autonomous mobile agents (e.g., robots) so as to jointly minimize travel time and energy consumption in the presence of multiple spatio-temporal constraints on these trajectories. In addition to state and input constraints, we impose spatial equality and temporal inequality constraints viewed as interior-point constraints. We address this problem by first identifying the structure of the optimal agent controllable acceleration profile and showing that it is characterized by several parameters subsequently used for trajectory design optimization. Therefore, the infinite dimensional optimal control problem is transformed into a finite dimensional parametric optimization problem. The proposed algorithm is applied to the eco-driving problem of autonomous vehicles approaching multiple signalized intersections. We include simulation results to show quantitatively the advantages of the proposed solution.

Finite element methods for one dimensional elliptic distributed optimal control problems with pointwise constraints on the derivative of the state

We investigate $C^1$ finite element methods for one dimensional elliptic distributed optimal control problems with pointwise constraints on the derivative of the state formulated as fourth order variational inequalities for the state variable. For the problem with Dirichlet boundary conditions, we use an existing $H^{\frac52-\epsilon}$ regularity result for the optimal state to derive $O(h^{\frac12-\epsilon})$ convergence for the approximation of the optimal state in the $H^2$ norm. For the problem with mixed Dirichlet and Neumann boundary conditions, we show that the optimal state belongs to $H^3$ under appropriate assumptions on the data and obtain $O(h)$ convergence for the approximation of the optimal state in the $H^2$ norm.

Optimal Portfolio Choice with Path Dependent Labor Income: the Infinite Horizon Case

We consider an infinite horizon portfolio problem with borrowing constraints, in which an agent receives labor income which adjusts to financial market shocks in a path dependent way. This path dependency is the novelty of the model and leads to an infinite dimensional stochastic optimal control problem. We solve the problem completely and find explicitly the optimal controls in feedback form. This is possible because we are able to find an explicit solution to the associated infinite dimensional HJB equation, even if state constraints are present. To the best of our knowledge, this is the first infinite dimensional generalization of Merton's optimal portfolio problem for which explicit solutions can be found. The explicit solution allows us to study the properties of optimal strategies and discuss their financial implications.

Optimal Portfolio Choice with Path Dependent Labor Income: The Infinite Horizon Case

We consider an infinite horizon portfolio problem with borrowing constraints, in which an agent receives labor income which adjusts to financial market shocks in a path dependent way. This path-dependency is the novelty of the model, and leads to an infinite dimensional stochastic optimal control problem. We solve the problem completely, and find explicitly the optimal controls in feedback form. This is possible because we are able to find an explicit solution to the associated infinite dimensional Hamilton-Jacobi-Bellman (HJB) equation, even if state constraints are present. To the best of our knowledge, this is the first infinite dimensional generalization of Merton's optimal portfolio problem for which explicit solutions can be found. The explicit solution allows us to study the properties of optimal strategies and discuss their financial implications.

Verification theorems for stochastic optimal control problems in Hilbert spaces by means of a generalized Dynkin formula

Verification theorems are key results to successfully employ the dynamic programming approach to optimal control problems. In this paper, we introduce a new method to prove verification theorems for infinite dimensional stochastic optimal control problems. The method applies in the case of additively controlled Ornstein–Uhlenbeck processes, when the associated Hamilton–Jacobi–Bellman (HJB) equation admits a mild solution (in the sense of [J. Differential Equations 262 (2017) 3343–3389]). The main methodological novelty of our result relies on the fact that it is not needed to prove, as in previous literature (see, e.g., [Comm. Partial Differential Equations 20 (1995) 775–826]), that the mild solution is a strong solution, that is, a suitable limit of classical solutions of approximating HJB equations. To achieve the goal, we prove a new type of Dynkin formula, which is the key tool for the proof of our main result.

Solving Stochastic LQR Problems by Polynomial Chaos

We consider the infinite dimensional stochastic linear quadratic optimal control problem for the infinite horizon case. We provide a numerical framework for solving this problem using a polynomial chaos expansion approach. By applying the method of chaos expansions to the state equation, we obtain a system of deterministic partial differential equations in terms of the coefficients of the state and the control variables. We set up a control problem for each equation, which results in a set of infinite horizon deterministic linear quadratic regulator problems. We prove the optimality of the solution expressed in terms of the expansion of these coefficients compared to the direct approach. We perform numerical experiments which validate our approach and compare the finite and infinite horizon case.

Deep Learning-Based BSDE Solver for Libor Market Model with Applications to Bermudan Swaption Pricing and Hedging

The Libor market model, also known as the BGM Model, is a term structure model of interest rates. It is widely used for pricing interest rate derivatives, especially Bermudan swaptions, and other exotic Libor callable derivatives. For numerical implementation the pricing of derivatives with Libor market models is mainly carried out with Monte Carlo simulation. The PDE grid approach is not particularly feasible due to the “Curse of Dimensionality”. The standard Monte Carlo method for American swaption pricing more or less uses regression to estimate the expected value as a linear combination of basis functions as demonstrated in the classical paper by Longstaff and Schwartz [Longstaff01]. However, [Longstaff01] only provides the lower bound for American option price. Another complexity arises from applying Monte Carlo simulation is the computation of the sensitivities of the option, the so-called “Greeks”, which are fundamental for a trader's hedging activity. Recently, an alternative numerical method based on deep learning and backward stochastic differential equations (BSDEs) appeared in quite a few researches [Weinan17, Jiequn17]. For European style options the feedforward deep neural networks (DNN) show not only feasibility but also efficiency to obtain both prices and numerical Greeks. The standard LMM implementation requires dimension five or higher in factor space even after PCA, which cannot be solved by traditional PDE solvers, such as finite differences or finite elements methods. In this paper, a new backward DNN solver is proposed for Bermudan swaptions. Our approach is representing financial pricing problems in the form of high dimensional stochastic optimal control problems, FBSDEs, or equivalent PDEs. We demonstrate that using backward DNN the high-dimension Bermudan swaption pricing and hedging can be solved effectively and efficiently. A comparison between Monte Carlo simulation and the new method for pricing vanilla interest rate options manifests the superior performance of the new method. We then use this method to calculate prices and Greeks of Bermudan swaptions as a prelude for other Libor callable derivatives.

Mixed and discontinuous finite volume element schemes for the optimal control of immiscible flow in porous media

In this article we introduce a family of discretisations for the numerical approximation of optimal control problems governed by the equations of immiscible displacement in porous media. The proposed schemes are based on mixed and discontinuous finite volume element methods in combination with the optimise-then-discretise approach for the approximation of the optimal control problem, leading to nonsymmetric algebraic systems, and employing minimum regularity requirements. Estimates for the error (between a local reference solution of the infinite dimensional optimal control problem and its hybrid mixed/discontinuous approximation) measured in suitable norms are derived, showing optimal orders of convergence.

Sensitivity analysis of the value function for infinite dimensional optimal control problems and its relation to Riccati equations

First and second order sensitivities of the value function in optimal control problems constrained by semilinear parabolic equations with respect to the initial condition are considered. Sufficient conditions are given which allow to express these sensitivities in terms of the adjoint equation and a suitably defined Riccati equation.

Turnpike Properties and Strict Dissipativity for Discrete Time Linear Quadratic Optimal Control Problems

We analyze the connection between turnpike behaviors and strict dissipativity properties for discrete time finite dimensional linear quadratic optimal control problems. We first use strict dissipativity as a sufficient condition for the turnpike property. Next, we characterize strict dissipativity and the newly introduced property of strict pre-dissipativity in terms of the system matrices related to the linear quadratic problem. These characterizations then lead to new necessary conditions for the turnpike properties under consideration, and thus eventually to necessary and sufficient conditions in terms of spectral criteria and matrix inequalities. One of the key novelties which distinguishes the results in the present paper from earlier ones on linear quadratic optimal control problems is the consideration of state and input constraints.

Method of evolving junctions: A new approach to optimal control with constraints

We propose a new strategy, called method of evolving junctions (MEJ), to compute the solutions for a class of optimal control problems with constraints on both state and control variables. Our main idea is that by leveraging the geometric structures of the optimal solutions, we recast the infinite dimensional optimal control problem into an optimization problem depending on a finite number of points, called junctions. Then, using a modified gradient flow method, whose dimension can change dynamically, we find local solutions for the optimal control problem. We also employ intermittent diffusion, a global optimization method based on stochastic differential equations, to obtain the global optimal solution. We demonstrate, via a numerical example, that MEJ can effectively solve path planning problems in dynamical environments.