Infinite-horizon Control Problem Research Articles

A two-player game representation for a class of infinite horizon control problems under state constraints

A two-player game representation for a class of infinite horizon control problems under state constraints

Correction to: Necessary Conditions in Infinite-Horizon Control Problems That Need No Asymptotic Assumptions

Correction to: Necessary Conditions in Infinite-Horizon Control Problems That Need No Asymptotic Assumptions

Necessary Conditions in Infinite-Horizon Control Problems that Need no Asymptotic Assumptions

Necessary Conditions in Infinite-Horizon Control Problems that Need no Asymptotic Assumptions

On the Optimality and Convergence Properties of the Iterative Learning Model Predictive Controller

In this technical article, we analyze the performance improvement and optimality properties of the learning model predictive control (LMPC) strategy for linear deterministic systems. The LMPC framework is a policy iteration scheme where closed-loop trajectories are used to update the control policy for the next execution of the control task. We show that, when a linear independence constraint qualification (LICQ) condition holds, the LMPC scheme guarantees strict iterative performance improvement and optimality, meaning that the closed-loop cost evaluated over the entire task converges asymptotically to the optimal cost of the infinite-horizon control problem. Compared to previous works, this sufficient LICQ condition can be easily checked, it holds for a larger class of systems and it can be used to adaptively select the prediction horizon of the controller, as demonstrated by a numerical example.

Optimal Singular Control Problem in Infinite Horizon for Stochastic Processes with Regime-Switching

We develop the compactification method to prove the existence of an optimal control for multidimensional stochastic processes with regime-switching. We consider separately the situations with and without state constraints. Three kinds of control policies are studied in this work: the classical controls on the drifts, the singular controls, and the controls on the transition rates of switching processes. Based on the existence of the optimal controls, the dynamic programming principle is also established.

Representation of Weak Solutions of Convex Hamilton–Jacobi–Bellman Equations on Infinite Horizon

In the present paper, it is provided a representation result for the weak solutions of a class of evolutionary Hamilton-Jacobi-Bellman equations on infinite horizon, with Hamiltonians measurable in time and fiber convex. Such Hamiltonians are associated with a - faithful - representation, namely involving two functions measurable in time and locally Lipschitz in the state and control. Our results concern the recovering of a representation of convex Hamiltonians under a relaxed assumption on the Fenchel transform of the Hamiltonian with respect to the fiber. We apply them to investigate the uniqueness of weak solutions, vanishing at infinity, of a class of time-dependent Hamilton-Jacobi-Bellman equations. Assuming a viability condition on the domain of the aforementioned Fenchel transform, these weak solutions are regarded as an appropriate value function of an infinite horizon control problem under state constraints.

Hamilton\u2013Jacobi equations for optimal control on multidimensional junctions with entry costs

We consider an infinite horizon control problem for dynamics constrained to remain on a multidimensional junction with entry costs. We derive the associated system of Hamilton–Jacobi equations (HJ), prove the comparison principle and that the value function of the optimal control problem is the unique viscosity solution of the HJ system. This is done under the usual strong controllability assumption and also under a weaker condition, coined ‘moderate controllability assumption’.

Regulator Problems on Unbounded Domains

In this paper, we consider a class of infinite horizon variational and control problems arising from economics, quantum mechanics, and stabilization. Herein, we assume that the objective is of regulator type. The problem setting implies a weighted Sobolev space as the state space. For this class of problems, we establish necessary optimality conditions in a form of a Pontryagin type maximum principle. A duality concept of convex analysis is provided and used to find sufficient optimality conditions and to motivate a dual approximation scheme. We apply the theoretical results to find an asymptotically stabilizing control for a linearized Lotka–Volterra type system.

Coupled State-Dependent Riccati Equation Control for Continuous Time Nonlinear Mechatronics Systems

This paper considers a novel coupled state-dependent Riccati equation (SDRE) approach for systematically designing nonlinear quadratic regulator (NLQR) and H∞ control of mechatronics systems. The state-dependent feedback control solutions can be obtained by solving a pair of coupled SDREs, guaranteeing nonlinear quadratic optimality with inherent stability property in combination with robust L2 type of disturbance reduction. The derivation of this control strategy is based on Nash's game theory. Both finite and infinite horizon control problems are discussed. An under-actuated robotic system, Furuta rotary pendulum, is used to examine the effectiveness and robustness of this novel nonlinear control approach.

The effects of technological shocks in an optimal goodwill model with a random product life cycle

We consider the optimal goodwill control problem in a segmented market where the length of the product life cycle is affected by unpredictable technological turbulence. In order to maximize profit over a random time horizon, a company controls the marketing efforts directed to each market segment. Assuming an exponential distribution for the product life cycle, we modify the optimal goodwill model into the infinite time horizon control problem. Based on the semigroup approach, we prove the existence and uniqueness of the optimal solution. We formulate optimality conditions for the problem and we prove the existence of a stationary long-run equilibrium. Next, we construct a numerical algorithm to find the optimal solution. Finally, we examine several scenarios of optimal marketing strategies.

Necessary stochastic maximum principle for dissipative systems on infinite time horizon

We develop a stochastic maximum principle for a finite-dimensional stochastic control problem in infinite horizon under a polynomial growth and joint monotonicity assumption on the coefficients. The second assumption generalizes the usual one in the sense that it is formulated as a joint condition for the drift and the diffusion term. The main difficulties concern the construction of the first and second order adjoint processes by solving backward equations on an unbounded time interval. The first adjoint process is characterized as a solution to a backward SDE, which is well-posed thanks to a duality argument. The second one can be defined via another duality relation written in terms of the Hamiltonian of the system and linearized state equation. Some known models verifying the joint monotonicity assumption are discussed as well.

Deterministic impulse control problems: Two discrete approximations of the quasi-variational inequality

In this paper, we study a deterministic infinite horizon, mixed continuous and impulse control problem in Rn, with general impulses, and cost of impulses. We assume that the cost of impulses is a positive function. We prove that the value function of the control problem is the unique viscosity solution of the related first order Hamilton–Jacobi quasi-variational inequality.11QVI. We then propose time discretization schemes of this QVI, where we consider two approximations of the “Hamiltonian hH”, including a natural one. We prove that the approximate value function uh exists, that it is the unique solution of the approximate QVI and that it forms a uniformly bounded and uniformly equicontinuous family. We also prove that the approximate value function converges locally uniformly, towards the value function of the control problem, when the discretization step h goes to zero; the rate of convergence is proved to be in hσ, where 0<σ<1/2.

A Bellman Approach for Regional Optimal Control Problems in $\mathbb{R}^N$

This article is a continuation of a previous work where we studied infinite horizon control problems for which the dynamic, running cost, and control space may be different in two half-spaces of some Euclidian space $\mathbb{R}^N$. In this article we extend our results in several directions: (i) to more general domains; (ii) to consideration of finite horizon control problems; (iii) to weakening the controllability assumptions. We use a Bellman approach and our main results are to identify the right Hamilton--Jacobi--Bellman equation (and, in particular, the right conditions to be put on the interfaces separating the regions where the dynamic and running cost are different) and to provide the maximal and minimal solutions, as well as conditions for uniqueness. We also provide stability results for such equations.

Necessity of Vanishing Shadow Price in Infinite Horizon Control Problems

This paper refines the necessary optimality conditions for uniformly overtaking optimal control on infinite horizon in the free end case. This condition is applicable to general non-stationary systems and the optimal objective value is not necessarily finite. In the papers of S.M. Aseev, A.V. Kryazhimskii, V.M. Veliov, K.O. Besov there was suggested a boundary condition for equations of the Pontryagin Maximum Principle. Each optimal process corresponds to a unique solution satisfying the boundary condition. Following A. Seierstad’s idea, in this paper we prove a more general geometric version of that boundary condition. We show that this condition is necessary for uniformly overtaking optimal control on infinite horizon in the free end case. A number of assumptions under which this condition selects a unique Lagrange multiplier is obtained. Some examples are discussed.

О необходимых краевых условиях для сильно оптимального управления в задачах на бесконечном промежутке

О необходимых краевых условиях для сильно оптимального управления в задачах на бесконечном промежутке

On the Existence of a Classical Optimal Solution and of an Almost Strongly Optimal Solution for an Infinite-Horizon Control Problem

We consider an infinite-horizon optimal control problem with the cost functional described either by an integral over an unbounded interval (a Lebesgue integral) or by a limit of integrals (an improper Lebesgue integral). We prove some theorems on the existence of solutions to such problems. The proofs are based on appropriate lower closure theorems and some extensions of Olech’s theorem on the lower semicontinuity of an integral functional; these extensions cover the cases of functionals described by an integral over an unbounded interval and by a limit of integrals.

A reduction of topological infinite-horizon optimization to periodic optimization in a class of compact 2-manifolds

We consider a general class of infinite-horizon optimization problems, where the phase space is a two dimensional manifold satisfying the Jordan curve theorem, and the set of feasible curves satisfies general conditions similar to, but weaker than the solution set of a control system. We verify the existence of an optimal periodic solution for this class of problems. Applying this result we obtain two new results: we generalize a previous result for discounted infinite-horizon control problems, and provide a Poincaré–Bendixson type result for continuous generalized semi-flows.

Linearization Techniques for Controlled Piecewise Deterministic Markov Processes; Application to Zubov’s Method

We aim at characterizing domains of attraction for controlled piecewise deterministic processes using an occupational measure formulation and Zubov’s approach. Firstly, we provide linear programming (primal and dual) formulations of discounted, infinite horizon control problems for PDMPs. These formulations involve an infinite-dimensional set of probability measures and are obtained using viscosity solutions theory. Secondly, these tools allow to construct stabilizing measures and to avoid the assumption of stability under concatenation for controls. The domain of controllability is then characterized as some level set of a convenient solution of the associated Hamilton-Jacobi integral-differential equation. The theoretical results are applied to PDMPs associated to stochastic gene networks. Explicit computations are given for Cook’s model for gene expression.

Applications des EDSR à horizon infini à un problème de contrôle impulsionnel

We study an impulse control problem in infinite horizon. To solve this problem, we extend to the infinite horizon case results of double barrier reflected backward stochastic differential equations. The properties of the Snell envelope reduce our problem to the existence of a pair of continuous processes.

Interface feedback control stabilization of a nonlinear fluid–structure interaction

We consider a model of fluid–structure interaction in a bounded domain Ω ∈ R n , n = 2 , where Ω is comprised of two open adjacent sub-domains occupied, respectively, by the solid and the fluid. This leads to a study of the Navier–Stokes equation coupled on the boundary with the dynamic system of elasticity. We shall consider models where the elastic body exhibits small but rapid oscillations. These are established models arising in engineering applications when the structure is immersed in a viscous flow of liquid. Questions related to the stability of finite energy solutions are of paramount interest. It was shown in Lasiecka and Lu (2011) [14] that all data of finite energy produce solutions whose energy converges strongly to zero. The cited result holds under “partial flatness” geometric condition whose role is to control the effects of the pressure in the NS equation. Related conditions has been used in Avalos and Triggiani (2008) [23] for the analysis of the linear model. The goal of the present work is to study uniform stability of all finite energy solutions corresponding to nonlinear interaction. This particular question, of interest in its own rights, is also a necessary preliminary step for the analysis of optimal control strategies arising in infinite-horizon control problems associated with the structure. It is shown in this paper that a stress type feedback control applied on the interface of the structure produces solutions whose energy is exponentially stable.