Ergodic Problem Research Articles

On an Ergodic Two-Sided Singular Control Problem

Motivated by applications in natural resource management, risk management, and finance, this paper is focused on an ergodic two-sided singular control problem for a general one-dimensional diffusion process. The control is given by a bounded variation process. Under some mild conditions, the optimal reward value as well as an optimal control policy are derived by the vanishing discount method. Moreover, the Abelian and Cesàro limits are established. Then a direct solution approach is provided at the end of the paper.

Sum-Rate-Optimal Statistical Precoding for FDD Massive MIMO Downlink With Deterministic Equivalents

Statisticalprecoding is considered as a promising technique to release the channel state information (CSI) acquisition overhead. This article investigates a linear precoder design for frequency-division duplexing (FDD) massive MIMO downlink with only statistical CSI. We use a beam-based statistical channel model to capture the spatial correlation characteristics of the channels. The objective of the precoder design is to maximize the ergodic sum-rate under total power constraint. Based on the minorize-maximize (MM) algorithm, a stationary solution of the ergodic sum-rate maximization problem can be obtained. The stationary solution is shown to be the same as the optimal solution to a stochastic weighted minimum mean square error (SWMMSE) problem. Further, we establish the approximations for rate expressions with deterministic equivalent. The deterministic equivalents of ergodic rates are only related to the precoding matrices and statistical CSI. According to these closed-form rate expressions, we propose a linear precoder design algorithm and obtain tractable expressions for precoding matrices. Numerical comparisons with the existing precoding approach demonstrate the significant advantages of the developed algorithm.

Optimal Power Allocation for Achieving Secure Green Cognitive Radio Networks

In cognitive radio networks, wireless nodes adapt to the surrounding radio environment and utilize the spectrum of licensed users. The cognitive radio environment is dynamic, and wireless channels are accessible by both legitimate and illegitimate users. Therefore, maintaining the security of cognitive radio networks is a challenging task, which must be addressed thoroughly. Further, with the recent exponential surge in wireless nodes and associated high data rate requirements, energy consumption is also growing at an unprecedented rate. Hence, energy efficiency becomes an important metric that must be considered in the design of future wireless networks. Accordingly, by considering the great ecological and economic benefits of green wireless networks, this work focus on energy-efficient resource allocation in secure cognitive radio networks. Since physical-layer security is an emerging technique that improves the security of communication devices, in this paper, an ergodic secure energy efficiency problem for a cognitive radio network is formulated with a primary user, a secondary user, and an eavesdropper. As the formulated problem is non-convex, a concave lower bound is applied to transform the original non-convex problem into a convex one. Further, by adopting the fractional programming and dual decomposition techniques, optimal power allocation strategies are obtained with the aim of maximizing the ergodic secure energy efficiency of the secondary user with constraints on the average interference power and average transmit power. Numerical examples are used to demonstrate the effectiveness of the proposed algorithm.

Optimal Control with Partially Observed Regime Switching: Discounted and Average Payoffs

We consider an optimal control problem with the discounted and average payoff. The reward rate (or cost rate) can be unbounded from above and below, and a Markovian switching stochastic differential equation gives the state variable dynamic. Markovian switching is represented by a hidden continuous-time Markov chain that can only be observed in Gaussian white noise. Our general aim is to give conditions for the existence of optimal Markov stationary controls. This fact generalizes the conditions that ensure the existence of optimal control policies for optimal control problems completely observed. We use standard dynamic programming techniques and the method of hidden Markov model filtering to achieve our goals. As applications of our results, we study the discounted linear quadratic regulator (LQR) problem, the ergodic LQR problem for the modeled quarter-car suspension, the average LQR problem for the modeled quarter-car suspension with damp, and an explicit application for an optimal pollution control.

Analytical and numerical solutions to ergodic control problems arising in environmental management

Environmental management optimizing a long‐run objective is an ergodic control problem whose resolution can be achieved by solving an associated non‐local Hamilton–Jacobi–Bellman (HJB) equation having an effective Hamiltonian. Focusing on sediment storage management as a modern engineering problem, we formulate, analyze, and compute a new ergodic control problem under discrete observations: a simple but non‐trivial mathematical problem. We give optimality and comparison results of the corresponding HJB equation having unique non‐smoothness and discontinuity. To numerically compute HJB equations, we propose a new fast‐sweep method resorting to neither pseudo‐time integration nor vanishing discount. The optimal policy and the effective Hamiltonian are then computed simultaneously. Convergence rate of numerical solutions is computationally analyzed. An advanced robust control counterpart where the dynamics involve uncertainties is also numerically considered.

Regularity of calibrated sub-actions for circle expanding maps and Sturmian optimization

AbstractIn this short and elementary note, we study some ergodic optimization problems for circle expanding maps. We first make an observation that if a function is not far from being convex, then its calibrated sub-actions are closer to convex functions in a certain effective way. As an application of this simple observation, for a circle doubling map, we generalize a result of Bousch saying that translations of the cosine function are uniquely optimized by Sturmian measures. Our argument follows the mainline of Bousch’s original proof, while some technical part is simplified by the observation mentioned above, and no numerical calculation is needed.

Sharp estimates of the generalized principal eigenvalue for superlinear viscous Hamilton–Jacobi equations with inward drift

This paper is concerned with the ergodic problem for viscous Hamilton–Jacobi equations with superlinear Hamiltonian, inward-pointing drift, and positive potential function which vanishes at infinity. Assuming some radial symmetry of the drift vector field and the potential function outside a large ball, we obtain sharp estimates of the generalized principal eigenvalue with respect to a perturbation of the potential function. We also specify the necessary and sufficient condition so that the spectral function contains a plateau.

Optimal Ergodic Harvesting under Ambiguity

We consider an ergodic harvesting problem with model ambiguity that arises from biology. To account for the ambiguity, the problem is constructed as a stochastic game with two players: the decision-maker (DM) chooses the `best' harvesting policy and an adverse player chooses the `worst' probability measure. The main result is establishing an optimal strategy (also referred to as a control) of the DM and showing that it is a threshold policy. The optimal threshold and the optimal payoff are obtained by solving a free-boundary problem emerging from the Hamilton--Jacobi--Bellman (HJB) equation. As part of the proof, we fix a gap that appeared in the HJB analysis of [Alvarez and Hening, {\em Stochastic Process. Appl.}, 2019], a paper that analyzed the risk-neutral version of the ergodic harvesting problem. Finally, we study the dependence of the optimal threshold and the optimal payoff on the ambiguity parameter and show that if the ambiguity goes to 0, the problem converges, to the risk-neutral problem.

Exterior Dirichlet and Neumann Problems and the Linked Ergodic Inverse Problems in the Entire Space

The paper is mainly concerned with the interconnection of the boundary behaviour of the solutions of the exterior Dirichlet and Neumann problems of harmonic analysis for the unit disk in {mathbb {R}}^2 and the unit ball in {mathbb {R}}^3 with the corresponding behaviour of the associated ergodic inverse problems for the entire space. The basis is the theory of semigroups of linear operators mapping a Banach space X into itself. The classical one-parameter theory for semigroups applies in the present particular applications, actually for X= L^2_{2pi } in case of the unit disk, and X=L^2(S) in the three dimensional setting, S being the unit sphere in {mathbb {R}}^3. Another tool is a Drazin-like inverse operator B for the infinitesimal generator A of a semigroup that arises naturally in ergodic theory. This operator B is a closed, not necessarily bounded, operator. It was introduced in a paper with Butzer and Westphal (Indiana Univ Math J 20:1163–1174, 1970/1971) and extended to a generalized setting with Butzer and Koliha (J Oper Theory 62:297–326, 2009).

Mean-Field Limit for a Class of Stochastic Ergodic Control Problems

Related DatabasesWeb of Science You must be logged in with an active subscription to view this.Article DataHistorySubmitted: 23 October 2020Accepted: 18 October 2021Published online: 14 February 2022Keywordsmean-field control, ergodic optimal control, McKean--Vlasov limit, de Finetti's theorem, strong Kac's chaos, convergence of probability measures on path spaceAMS Subject Headings49N80, 60H10, 60F15Publication DataISSN (print): 0363-0129ISSN (online): 1095-7138Publisher: Society for Industrial and Applied MathematicsCODEN: sjcodc

On ergodic control problem for viscous Hamilton–Jacobi equations for weakly coupled elliptic systems

In this article, we study ergodic problems in the whole space RN for weakly coupled systems of viscous Hamilton-Jacobi equations with coercive right-hand sides. The Hamiltonians are assumed to have a fairly general structure, and the switching rates need not be constant. We prove the existence of a critical value ▪ such that the ergodic eigenvalue problem has a solution for every ▪ and no solution for ▪. Moreover, the existence and uniqueness of non-negative solutions corresponding to the value ▪ are also established. We also exhibit the implication of these results to the ergodic optimal control problems of controlled switching diffusions.

Ergodic risk-sensitive control of Markov processes on countable state space revisited

We consider a large family of discrete and continuous time controlled Markov processes and study an ergodic risk-sensitive minimization problem. Under a blanket stability assumption, we provide a complete analysis to this problem. In particular, we establish uniqueness of the value function and verification result for optimal stationary Markov controls, in addition to the existence results. We also revisit this problem under a near-monotonicity condition but without any stability hypothesis. Our results also include policy improvement algorithms both in discrete and continuous time frameworks.

Quasistationary distributions and ergodic control problems

We introduce and study the basic properties of two ergodic stochastic control problems associated with the quasistationary distribution (QSD) of a diffusion process X relative to a bounded domain. The two problems are in some sense dual, with one defined in terms of the generator associated with X and the other in terms of its adjoint. Besides proving wellposedness of the associated Hamilton–Jacobi–Bellman equations, we describe how they can be used to characterize important properties of the QSD. Of particular note is that the QSD itself can be identified, up to normalization, in terms of the cost potential of the control problem associated with the adjoint.

The large time profile for Hamilton–Jacobi–Bellman equations

Here, we study the large-time limit of viscosity solutions of the Cauchy problem for second-order Hamilton–Jacobi–Bellman equations with convex Hamiltonians in the torus. This large-time limit solves the corresponding stationary problem, sometimes called the ergodic problem. This problem, however, has multiple viscosity solutions and, thus, a key question is which of these solutions is selected by the limit. Here, we provide a representation for the viscosity solution to the Cauchy problem in terms of generalized holonomic measures. Then, we use this representation to characterize the large-time limit in terms of the initial data and generalized Mather measures. In addition, we establish various results on generalized Mather measures and duality theorems that are of independent interest.

Ergodic inventory control with diffusion demand and general ordering costs

In this work, we consider a continuous-time inventory system where the demand process follows an inventory-dependent diffusion process. The ordering cost of each order depends on the order quantity and is given by a general function, which is not even necessarily continuous and monotone. By applying a lower bound approach together with a comparison theorem, we show the global optimality of an (s,S) policy for this ergodic inventory control problem.

Remarks on the vanishing discount problem for infinite systems of Hamilton–Jacobi–Bellman equations

Abstract This paper is concerned with the asymptotic analysis of infinite systems of weakly coupled stationary Hamilton–Jacobi–Bellman equations as the discount factor tends to zero. With a specific Hamiltonian, we show the convergence of the solution and prove the solvability of the corresponding ergodic problem.

A tale of two rhythms: Locked clocks and chaos in biology.

The fundamental mechanisms that control and regulate biological organisms exhibit a surprising level of complexity. Oscillators are perhaps the simplest motifs that produce time-varying dynamics and are ubiquitous in biological systems. It is also known that such biological oscillators interact with each other-for instance, circadian oscillators affect the cell cycle, and somitogenesis clock proteins in adjacent cells affect each other in developing embryos. Therefore, it is vital to understand the effects that can emerge from non-linear interaction between oscillations. Here, we show how oscillations typically arise in biology and take the reader on a tour through the great variety in dynamics that can emerge even from a single pair of coupled oscillators. We explain how chaotic dynamics can emerge and outline the methods of detecting this in experimental time traces. Finally, we discuss the potential role of such complex dynamical features in biological systems.

Ergodic control of diffusions with random intervention times

AbstractWe study an ergodic singular control problem with constraint of a regular one-dimensional linear diffusion. The constraint allows the agent to control the diffusion only at the jump times of an independent Poisson process. Under relatively weak assumptions, we characterize the optimal solution as an impulse-type control policy, where it is optimal to exert the exact amount of control needed to push the process to a unique threshold. Moreover, we discuss the connection of the present problem to ergodic singular control problems, and illustrate the results with different well-known cost and diffusion structures.

Ergodic pairs for degenerate pseudo Pucci's fully nonlinear operators

We study the ergodic problem for fully nonlinear operators which may be singular or degenerate when at least one of the components of the gradient vanishes. We extend here the results in [ 16 ], [ 10 ], [ 24 ].

Structure of intergenerational risk-sharing plans: optimality and fairness

In this paper, we derive optimal designs for a stylized Intergenerational Risk Sharing (IRS) pension plan. We study a Defined Ambition plan under which both contributions and pension benefits are adjusted based on the funding level. Our objective function focuses on the stability of members' lifetime consumption, both in the contribution and benefit phases, formulating the optimization as an ergodic control problem. We illustrate the drawbacks of unconstrained optimization and demonstrate the importance of including regulatory requirements for the sake of fairness across generations. Our results show that a linear risk sharing protocol is transparent, and with reasonable constraints on the funding level, can deliver a risk sharing plan that is fair with respect to intergenerational risk transfers, and is sustainable.