Ergodic Optimal Control Research Articles

A numerical method for ergodic optimal control of switching diffusions with reflection

The ergodic or long run average cost control problem for diffusions is one of the classical problems of stochastic control that still eludes a completely satisfactory treatment. This is certainly true for the setting in which the systems to be controlled is modeled by the solution of a switching stochastic differential equation with reflection (SSDER). In this paper, we advance in this rather difficult problem by setting forth preliminary results for the unidimensional case. Besides carving out a numerical method, our treatment of the ergodic control in this scenario straddles issues of existence and uniqueness of solution of the SSDER and a verification theorem for the associated HJB equation. We conclude by illustrating the effectiveness of the method considering the control of energy consumption in a large parallel processing computer system composed of one queue and several processing stations.

Mean-Field Limit for a Class of Stochastic Ergodic Control Problems

Related DatabasesWeb of Science You must be logged in with an active subscription to view this.Article DataHistorySubmitted: 23 October 2020Accepted: 18 October 2021Published online: 14 February 2022Keywordsmean-field control, ergodic optimal control, McKean--Vlasov limit, de Finetti's theorem, strong Kac's chaos, convergence of probability measures on path spaceAMS Subject Headings49N80, 60H10, 60F15Publication DataISSN (print): 0363-0129ISSN (online): 1095-7138Publisher: Society for Industrial and Applied MathematicsCODEN: sjcodc

On ergodic control problem for viscous Hamilton–Jacobi equations for weakly coupled elliptic systems

In this article, we study ergodic problems in the whole space RN for weakly coupled systems of viscous Hamilton-Jacobi equations with coercive right-hand sides. The Hamiltonians are assumed to have a fairly general structure, and the switching rates need not be constant. We prove the existence of a critical value ▪ such that the ergodic eigenvalue problem has a solution for every ▪ and no solution for ▪. Moreover, the existence and uniqueness of non-negative solutions corresponding to the value ▪ are also established. We also exhibit the implication of these results to the ergodic optimal control problems of controlled switching diffusions.

Additive Eigenvalue Problems of the Laplace Operator with the Prescribed Contact Angle Boundary Condition

Additive eigenvalue problem appears in ergodic optimal control or the homogenization of Hamilton–Jacobi equations. It has wide applications in several fields including computer science and then attracts the attention. In this paper, we consider the Poisson equations with the prescribed contact angle boundary condition and finally derive the existence and the uniqueness of the solution to the additive problem of the Laplace operator with the prescribed contact angle boundary condition.

Ergodic BSDE with unbounded and multiplicative underlying diffusion and application to large time behaviour of viscosity solution of HJB equation

We study ergodic backward stochastic differential equations (EBSDEs), for which the underlying diffusion is assumed to be multiplicative and of linear growth. The fact that the forward process has an unbounded diffusion is balanced with an assumption of weak dissipativity for its drift. Moreover, the forward equation is assumed to be non-degenerate. We study the existence and uniqueness of EBSDEs and we apply our results to an ergodic optimal control problem. In particular, we show the large time behaviour of viscosity solution of Hamilton–Jacobi–Bellman equation with an exponential rate of convergence when the underlying diffusion is multiplicative and unbounded.

Asymptotic harvesting of populations in random environments.

We consider the harvesting of a population in a stochastic environment whose dynamics in the absence of harvesting is described by a one dimensional diffusion. Using ergodic optimal control, we find the optimal harvesting strategy which maximizes the asymptotic yield of harvested individuals. To our knowledge, ergodic optimal control has not been used before to study harvesting strategies. However, it is a natural framework because the optimal harvesting strategy will never be such that the population is harvested to extinction-instead the harvested population converges to a unique invariant probability measure. When the yield function is the identity, we show that the optimal strategy has a bang-bang property: there exists a threshold [Formula: see text] such that whenever the population is under the threshold the harvesting rate must be zero, whereas when the population is above the threshold the harvesting rate must be at the upper limit. We provide upper and lower bounds on the maximal asymptotic yield, and explore via numerical simulations how the harvesting threshold and the maximal asymptotic yield change with the growth rate, maximal harvesting rate, or the competition rate. We also show that, if the yield function is [Formula: see text] and strictly concave, then the optimal harvesting strategy is continuous, whereas when the yield function is convex the optimal strategy is of bang-bang type. This shows that one cannot always expect bang-bang type optimal controls.

Ergodic BSDEs and related PDEs with Neumann boundary conditions under weak dissipative assumptions

We study a class of ergodic BSDEs related to PDEs with Neumann boundary conditions. The randomness of the driver is given by a forward process under weakly dissipative assumptions with an invertible and bounded diffusion matrix. Furthermore, this forward process is reflected in a convex subset of Rd not necessarily bounded. We study the link of such EBSDEs with PDEs and we apply our results to an ergodic optimal control problem.

Numerical solution of a minimax ergodic optimal control problem

AbstractIn this work we consider an L∞ minimax ergodic optimal control problem with cumulative cost. We approximate the cost function as a limit of evolutions problems. We present the associated Hamilton‐Jacobi‐Bellman equation and we prove that it has a unique solution in the viscosity sense. As this HJB equation is consistent with a numerical procedure, we use this discretization to obtain a procedure for the primitive problem. For the numerical solution of the ergodic version we need a perturbation of the instantaneous cost function. We give an appropriate selection of the discretization and penalization parameters to obtain discrete solutions that converge to the optimal cost. We present numerical results. (© 2008 WILEY‐VCH Verlag GmbH & Co. KGaA, Weinheim)

Optimal control of a one-dimensional storage process

We consider the discounted and ergodic optimal control problems related to a one-dimensional storage process. The existence and uniqueness of the corresponding Bellman equation and the regularity of the optimal value is established. Using the Bellman equation an optimal feedback control is constructed. Finally we show that under this optimal control the origin is reachable.