Infinite Horizon Optimization Problem Research Articles

Average Optimality in Nonhomogeneous Infinite Horizon Markov Decision Processes

We consider a nonhomogeneous stochastic infinite horizon optimization problem whose objective is to minimize the overall average cost per period of an infinite sequence of actions (average optimality). Optimal solutions to such problems will in general be nonstationary. Moreover, a solution that initially makes poor decisions, and then selects wisely thereafter, can be average optimal. However, we seek average optimal solutions with optimal short-term, as well as long-term, behavior. Our approach is to first transform our stochastic problem into one that is deterministic, using the standard device of formulating the problem as one of choosing a sequence of policies, as opposed to actions. Within this deterministic framework, states become probability distributions over the original stochastic states. Then, by weakening the notion of state reachability, and strengthening the notion of efficiency traditionally used in the deterministic framework, we prove that such efficient solutions exist and are average optimal, thus simultaneously exhibiting both optimal long- and short-run behavior. This deterministic view of the property of stochastic ergodicity offers the potential to relax the traditional conditions for average optimality that use coefficients of ergodicity, as well as the opportunity to strengthen the criterion of average optimality through the property of efficiency.

Bifurcations of optimal vector fields in the shallow lake model

The solution structure of the set of optimal solutions of the shallow lake problem, a problem of optimal pollution management, is studied as we vary the values of the system parameters: the natural resilience, the relative importance of the resource for social welfare and the future discount rate. We find parameter values at which qualitative changes occur. Using theoretical results on the bifurcations of the solution structure to infinite horizon optimization problems obtained earlier, we give a fairly complete bifurcation analysis of the shallow lake problem. In particular, we show how the increase of the discount rate affects the parameter regions where an oligotrophic steady state, corresponding to low pollution level, is globally stable or locally stable under optimal dynamics. Asymptotically, an increase of the discount rate can be offset with a proportional increase of the relative social weight of the resource.

Optimal solutions to the infinite horizon problems: Constructing the optimum as the limit of the solutions for the finite horizon problems

The limit of the solutions for the finite horizon problems is not always optimal for the infinite horizon problems. In this paper, we provide some special conditions that ensure its validity. It can be applied to constructing the optimal solutions for the undiscounted infinite horizon optimization problems. We also demonstrate a simple example to explain this structure.

Value and Policy Function Approximations in Infinite-Horizon Optimization Problems

Suboptimal solutions to infinite-horizon dynamic optimization problems with continuous state are considered. An underlying dynamical system determining the state transition between each stage and the next one is modelled via the constraints (xt, xt +1) ∈ D, t = 0, 1, …, where X is the set to which the state vector belongs and D ⊆ X × X is a correspondence. An error analysis is performed for two cases: approximation of the value function and approximation of the optimal policy function. Structural properties of the dynamic optimization problems are derived, allowing to restrict a priori the approximation to families of functions characterized by certain smoothness properties. The two approximation approaches are compared and the respective pros and cons are highlighted.

Limit of the solutions for the finite horizon problems as the optimal solution to the infinite horizon optimization problems

In this paper, we consider how to construct the optimal solutions for the undiscounted discrete time infinite horizon optimization problems. We present the conditions under which the limit of the solutions for the finite horizon problems is optimal among all attainable paths for the infinite horizon problem under two modified overtaking criteria, as well as the conditions under which it is the unique optimum under the sum-of-utilities criterion. The results are applied to a parametric example of a simple one-sector growth model to examine the impacts of discounting on the optimal path.

Optimal regulation of homogeneous systems

We present an offline numerical algorithm to generate a discontinuous feedback law to robustly regulate the origin of a continuous-time homogeneous system through sample and hold. The proposed feedback comes out of the solution of an infinite horizon optimization problem in discrete time and is in the form of a look-up table. We show that when applied to either chained systems or systems in power form the algorithm results in a closed loop whose origin is globally exponentially stable.

Optimization-Based Stabilization of Sampled-Data Nonlinear Systems via Their Approximate Discrete-Time Models

We present results on numerical regulator design for sampled-data nonlinear plants via their approximate discrete-time plant models. The regulator design is based on an approximate discrete-time plant model and is carried out either via an infinite horizon optimization problem or via a finite horizon with terminal cost optimization problem. In both cases, we discuss situations when the sampling period T and the integration period h used in obtaining the approximate discrete-time plant model are the same or they are independent of each other. We show that, using this approach, practical and/or semiglobal stability of the exact discrete-time model is achieved under appropriate conditions.

Construction of Stationary Markov Equilibria in a Strategic Market Game

This paper studies stationary noncooperative equilibria in an economy with fiat money, one nondurable commodity, countably many time-periods, no credit or futures market, and a measure space of agents—who may differ in their preferences and in the distributions of their (random) endowments. These agents are immortal, and hold fiat money as a means of hedging against the random fluctuations in their endowments of the commodity. In the aggregate, these fluctuations offset each other, and equilibrium prices are constant. We carry out an equilibrium analysis that focuses on distribution of wealth, on consumption, and on price formation. A careful analysis of the one-agent, infinite-horizon optimization problem, and of the invariant measure for the associated optimally controlled Markov chain, leads by aggregation to a stationary noncooperative or competitive equilibrium. This consists of a price for the commodity and of a distribution of wealth across agents which, under appropriate simple strategies for the agents, stay fixed from period to period and preserve the basic quantities of the model. We hope that, in future work, we shall be able to address additional features of the model treated here, such as borrowing and lending at appropriate (endogenously determined) interest rates, the endogenous production of commodity, overlapping generations of agents, and bankruptcy and treatment of creditors.

Discrete-Time Finite Horizon Approximation of Infinite Horizon Optimization Problems with Steady-State Invariance

The computation of large-scale nonlinear intertemporal optimization problems requires time aggregation. A procedure generally adopted is shown to introduce a dependency of the solution steady state to a specific choice of sequence of time intervals. The authors establish necessary and sufficient conditions to avoid this dependency. The result is a considerable improvement in the numerical accuracy of the time-aggregated approximation. The conditions apply to a broad class of models and prove useful in large-scale applied general-equilibrium modeling. This conclusion is highlighted by a comparison with a spectral-projection method using optimal orthogonal collocation as in K. L. Judd (1992). Copyright 1994 by The Econometric Society.

Conditions for the discovery of solution horizons

We present necessary and sufficient conditions for discrete infinite horizon optimization problems with unique solutions to be solvable. These problems can be equivalently viewed as the task of finding a shortest path in an infinite directed network. We provide general forward algorithms with stopping rules for their solution. The key condition required is that of weak reachability, which roughly requires that for any sequence of nodes or states, it must be possible from optimal states to reach states close in cost to states along this sequence. Moreover the costs to reach these states must converge to zero. Applications are considered in optimal search, undiscounted Markov decision processes, and deterministic infinite horizon optimization.

A Tie-Breaking Rule for Discrete Infinite Horizon Optimization

We study discrete infinite horizon optimization problems without the common assumption of a unique optimum. A method based on solution set convergence is employed for finding optimal initial decisions by solving finite horizon problems. This method is applicable to general discrete decision models that satisfy a weak reachability condition. The algorithm, together with a stopping rule, is applied to production planning and capacity expansion, and computational results are reported.

Existence Results and Finite Horizon Approximates for Infinite Horizon Optimization Problems

The paper deals with infinite horizon optimization problems. The existence of optimal solutions is obtained as a consequence of an asymptotic growth condition. We also exhibit finite horizon approximates that yield upper and lower bounds for the optimal values and whose optimal solutions converge to the long-term optimal trajectories.

Transversality Conditions for Some Infinite Horizon Discrete Time Optimization Problems

This paper considers a class of discrete-time, infinite-horizon optimization problems arising in economics. Necessary optimality conditions for such problems consist of an Euler equation and a tran...

Transversality Conditions for Some Infinite Horizon Discrete Time Optimization Problems

This paper considers a class of discrete-time, infinite-horizon optimization problems arising in economics. Necessary optimality conditions for such problems consist of an Euler equation and a transversality condition at infinity. Our concern here is with the latter condition. Previous results for this type of problem have required such restrictions as concavity and boundedness of the utility functions whose sum is being maximized. Here we derive a transversality condition that remains valid under considerably more general hypotheses. In doing so, we introduce methods that may also be applicable to other situations, such as optimization in continuous time.

Long-run behavior of constrained discrete-time linear dynamical systems

We consider a system of linear equations whose variablesy(t) ź Em describe an economy or process which is observed in succeeding discrete time periodst. With forms given for the equations relating the values of the state vectory(t) in succeeding periods and limiting the set of feasible choices ofy(t) in each period, conditions which determine the nature of feasible sequences of state vectors are given, and the characteristics ofy(t) astźź are analyzed. Some of the results are studied in the framework of specified infinite horizon optimization problems.

Convergence of the feasible region in infinite horizon optimization problems

Convergence of the feasible region in infinite horizon optimization problems