Infinite Horizon Optimal Control Problem Research Articles

Convergence of the Value Functions of Discounted Infinite Horizon Optimal Control Problems with Low Discount Rates

For autonomous, nonlinear, smooth optimal control systems on n-dimensional manifolds we investigate the relationship between the discounted and the average yield optimal value of infinite horizon problems. It is shown that the value functions of discounted problems converge to the value function of the average yield problem as the discount rate tends to zero, if there exist approximately optimal solutions satisfying some periodicity conditions. In general, the discounted value functions cannot be expected to converge, which is shown by a counterexample. A connection to geometric control theory is then made to establish a result of uniform convergence on compact subsets of the interior of control sets, if optimal trajectories do not leave a compact subset of the interior of these control sets.

Nonconvex and relaxed infinite-horizon optimal control problems

In this paper, we consider the Lagrange problem of optimal control defined on an unbounded time interval in which the traditional convexity hypotheses are not met. Models of this form have been introduced into the economics literature to investigate the exploitation of a renewable resource and to treat various aspects of continuous-time investment. An additional distinguishing feature in the models considered is that we do not assume a priori that the objective functional (described by an improper integral) is finite, and so we are led to consider the weaker notions of overtaking and weakly overtaking optimality. To treat these models, we introduce a relaxed optimal control problem through the introduction of chattering controls. This leads us naturally to consider the relationship between the original problem and the convexified relaxed problem. In particular, we show that the relaxed problem may be viewed as a limiting case for the original problem. We also present several examples demonstrating the applicability of our results.

Optimal controls for diffusion in Rd—A min-max max-min formula for the minimal cost growth rate

We study the infinite horizon optimal control problem for a system whose dynamics are described by the stochastic differential equation dx, = Wu u,) dt + 4x,) da,> x0 given. (1.1) Let e be the o-algebra generated by (j,jO G s G r; then an admissible control is an {*}t,, adapted process in R” which satisfies the constraint u, E Wx,), t 2 0. ‘In the last relation x + U(x) is a measurable set valued mapping from Rd to R”. To every control corresponds the cost flow

The Existence of Catching-Up Optimal Solutions for a Class of Infinite Horizon Optimal Control Problems with Time Delay

The optimal control of a system whose states are governed by a nonlinear autonomous Volterra integrodifferential equation with unbounded time interval is considered. Specifically, it is assumed that the delay occurs only in the state variable. The results obtained extend those of Brock and Haurie [Math. Oper. Res., 1 (1976), pp. 337–346] and Leizarowitz [Math. Oper. Res., 10 (1985), pp. 450–461]. In particular, it is shown that (under appropriate hypotheses) catching-up optimal solutions asymptotically approach a unique optimal steady state, and thus enjoy the so-called “turnpike” property found in the economics literature. By combining this result with an associated optimal control problem, the desired existence result is obtained. Furthermore, it is remarked that, in addition to extending these earlier works to the time-delay case, the results presented below utilize convexity, seminormality, and growth hypotheses that in some cases are weaker than those encountered in the above-mentioned papers.

An optimal control problem with a random stopping time

This paper deals with a stochastic optimal control problem where the randomness is essentially concentrated in the stopping time terminating the process. If the stopping time is characterized by an intensity depending on the state and control variables, one can reformulate the problem equivalently as an infinite-horizon optimal control problem. Applying dynamic programming and minimum principle techniques to this associated deterministic control problem yields specific optimality conditions for the original stochastic control problem. It is also possible to characterize extremal steady states. The model is illustrated by an example related to the economics of technological innovation.

On the existence of sporadically catching-up optimal solutions for infinite-horizon optimal control problems

In this paper, we extend the existence theory of Brock and Haurie concerning the existence of sporadically catching-up optimal solutions for autonomous, infinite-horizon optimal control problems. This notion of optimality is one of a hierarchy of types of optimality that have appeared in the literature to deal with optimal control problems whose cost functionals, described by an improper integral, either diverge or are unbounded below. Our results rely on the now classical convexity and seminormality hypotheses due to Cesari and are weaker than those assumed in the work of Brock and Haurie. An example is presented where our results are applicable, but those of the above-mentioned authors do not.

The controllability of infinite horizon optimal control problems

The controllability of infinite horizon optimal control problems

Existence of finitely optimal solutions for infinite-horizon optimal control problems

In this paper, we investigate the existence of finitely optimal solutions for the Lagrange problem of optimal control defined on [0, ∞) under weaker convexity and seminormality hypotheses than those of previous authors. The notion of finite optimality has been introduced into the literature as the weakest of a hierarchy of types of optimality that have been defined to permit the study of Lagrange problems, arising in mathematical economics, whose cost functions either diverge or are not bounded below. Our method of proof requires us to analyze the continuous dependence of finite-interval Lagrange problems with respect to a prescribed terminal condition. Once this is done, we show that a finitely optimal solution can be obtained as the limit of a sequence of solutions to a sequence of corresponding finite-horizon optimal control problems. Our results utilize the convexity and seminormality hypotheses which are now classical in the existence theory of optimal control.

An on-line procedure in discounted infinite-horizon stochastic optimal control

We consider the discrete-time, infinite-horizon optimal control problem with discounted cost. We propose a test to detect forecast/planning horizons, and derive an on-line procedure for solving the original problem.

On the existence of catching-up optimal solutions for Lagrange problems defined on unbounded intervals

In this paper, we are concerned with the question of the existence of optimal solutions for infinite-horizon optimal control problems of Lagrange type. In such problems, the objective or cost functional is described by an improper integral. As dictated by applications arising in mathematical economics, we do nota priori assume that this improper integral converges. This leads us to consider a weaker type of optimality, known as catching-up optimality. The results presented here utilize the classical convexity and seminormality conditions typically imposed in the existence theory for the case of finite intervals. These conditions are significantly weaker than those imposed by other authors; as a consequence, their existence results are contained as special cases of the results presented here. The method of proof utilizes the Caratheodory-Hamilton-Jacobi theory previously developed by the author for infinite-horizon optimal control problems.

A Carath�odory-Hamilton-Jacobi theory for infinite-horizon optimal control problems

In this paper, we extend Caratheodory's concept of equivalent variational problems to infinite-horizon optimal control problems. In such a setting, the usual concept of a minimum need not exist, and we therefore consider a weaker definition of optimality, known as catching up optimality. The extension presented here leads us to a Hamilton-Jacobi theory for infinite-horizon optimal control problems that closely parallels the classical work of Caratheodory as well as providing sufficient conditions for optimality. Finally, we show that the results given here subsume several previously known results as a special case.

Detecting planning horizons in deterministic infinite horizon optimal control

A deterministic infinite horizon optimal control problem with discounted cost is investigated. Under some assumptions we give a method to detect a planning horizon which is guaranteed to exist. When forward dynamic programming can be applied, the procedure does not require extra computations. An inventory control example is presented.

An existence theorem for discrete-time infinite-horizon optimal control problems

An existence theorem is given for a general class of deterministic, infinite-horizon, discrete-time optimal control problems. The hypotheses of the theorem are weak and can be easily verified. The objective function may be either a summation or supremum over time.

The determining set in the infinite horizon problem of optimal control

AbstractThis paper will study the adjoint vector in the infinite horizon optimal control problem. The approach used is to translate the adjoint vector, by means of the fundamental solution of the homogeneous variational equation, back to the initial time. Thus, the set of adjoint vectors ‘satisfying the maximum principle’ will be formed at the initial time. This set will be called the ‘determining set’ and will then be used to ascertain conditions of optimality. Properties, applications and an example of the determining set are given.

Detectability of linear discrete-time systems with stochastic parameters

Abstract The paper introduces the concept of mean square detectability and relates this to the recently introduced concept of mean square observability. It is shown that under appropriate mean square detectability and stabilizability conditions the infinite-horizon optimal control problem for the general case of linear discrete time systems and quadratic criteria, both with stochastic parameters which are statistically independent of time, has a unique solution when the control system is mean square stable. A simple necessary and sufficient condition, explicit in the system matrices, is given to determine if a system is mean square detectable. This condition also holds for deterministic systems to be detectable in the usual sense. The mean square detectability property coincides with the usual one if the parameters are deterministic.

On a discrete approximation of the Hamilton-Jacobi equation of dynamic programming

An approximation of the Hamilton-Jacobi-Bellman equation connected with the infinite horizon optimal control problem with discount is proposed. The approximate solutions are shown to converge uniformly to the viscosity solution, in the sense of Crandall-Lions, of the original problem. Moreover, the approximate solutions are interpreted as value functions of some discrete time control problem. This allows to construct by dynamic programming a minimizing sequence of piecewise constant controls.

Infinite horizon optimal control of linear discrete time systems with stochastic parameters

The infinite horizon optimal control problem is considered in the general case of linear discrete time systems and quadratic criteria, both with stochastic parameters which are independent with respect to time. A stronger stabilizability property and a weaker observability property than usual for deterministic systems are introduced. It is shown that the infinite horizon problem has a solution if the system has the first property. If in addition the problem has the second property the solution is unique and the control system is stable in the mean square sense. A simple necessary and sufficient condition, explicit in the system matrices, is given for the system to have the stronger stabilizability property. This condition also holds for deterministic systems to be stabilizable in the usual sense. The stronger stabilizability and weaker observability properties coincide with the usual ones if the parameters are deterministic.

Qualitative asymptotic synthesis in simple optimal control problems

Many infinite horizon optimal control problems in economics have scalar state and control variables. Under frequently invoked assumptions, it is sometimes very simple to tell whether the control in such a problem is increasing or decreasing in the state near equilibrium.

Nearest feasible paths in optimal control problems: Theory, examples, and counterexamples

Many infinite-horizon optimal control problems in management science and economics have optimal paths that approach some stationary level. Often, this path has the property of being the nearest feasible path to the stationary equilibrium. This paper obtains a simple multiplicative characterization for a single-state single-control problem to have this property. By using Green's theorem it is shown that the property is observed as long as the stationary level is sustainable by a feasible control. If not, the property is, in general, shown to be false. The paper concludes with an important theorem which states that even in the case of multiple equilibria, the optimal path is a nearest feasible path to one of them.

National settlement policies

In this paper we study optimal policies for a central planner interested in maximizing utility in an economy driven by a renewable resource. It is shown that the optimal consumption path is sustainable only when the intrinsic growth rate of the resource is greater than the social discount rate. The model is formulated as an infinite horizon optimal control problem. We deal with the mathematical details of the problem, develop a precise notion for optimality and establish the existence of optimal control at least when the condition for sustainability is met. We apply the appropriate version of the Pontryagin maximum principle and show a numerical simulation of the optimal feedback law. In the end we present the results along with physical interpretations.