Infinite Time Horizon Problem Research Articles

Numerical computation of the optimal vector field: Exemplified by a fishery model

Numerous optimal control models analyzed in economics are formulated as discounted infinite time horizon problems, where the defining functions are nonlinear as well in the states as in the controls. As a consequence solutions can often only be found numerically. Moreover, the long run optimal solutions are mostly limit sets like equilibria or limit cycles. Using these specific solutions a BVP approach together with a continuation technique is used to calculate the parameter dependent dynamic structure of the optimal vector field. We use a one dimensional optimal control model of a fishery to exemplify the numerical techniques. But these methods are applicable to a much wider class of optimal control problems with a moderate number of state and control variables.

Optimal dynamic pricing of inventories with stochastic demand and discounted criterion

We consider a continuous time dynamic pricing problem for selling a given number of items over a finite or infinite time horizon. The demand is price sensitive and follows a non-homogeneous Poisson process. We formulate this problem as to maximize the expected discounted revenue and obtain the structural properties of the optimal revenue function and optimal price policy by the Hamilton–Jacobi–Bellman (HJB) equation. Moreover, we study the impact of the discount rate on the optimal revenue function and the optimal price. Further, we extend the problem to the case with discounting and time-varying demand, the infinite time horizon problem. Numerical examples are used to illustrate our analytical results.

Single vehicle routing problems with a predefined customer sequence, compartmentalized load and stochastic demands

We consider the problem of finding the optimal routing of a single vehicle that delivers K different products to N customers according to a particular customer order. The demands of the customers for each product are assumed to be random variables with known distributions. Each product type is stored in its dedicated compartment in the vehicle. Using a suitable dynamic programming algorithm we find the policy that satisfies the demands of the customers with the minimum total expected cost. We also prove that this policy has a specific threshold-type structure. Furthermore, we investigate a corresponding infinite-time horizon problem in which the service of the customers does not stop when the last customer has been serviced but it continues indefinitely with the same customer order. It is assumed that the demands of the customers at different tours have the same distributions. It is shown that the discounted-cost optimal policy and the average-cost optimal policy have the same threshold-type structure as the optimal policy in the original problem. The theoretical results are illustrated by numerical examples.

Separation principle in the fractional Gaussian linear-quadratic regulator problem with partial observation

In this paper we solve the basic fractional analogue of the classical linear-quadratic Gaussian regulator problem in continuous-time with partial observation. For a controlled linear system where both the state and observation processes are driven by fractional Brownian motions, we describe explicitly the optimal control policy which minimizes a quadratic performance criterion. Actually, we show that a separation principle holds, i.e., the optimal control separates into two stages based on optimal filtering of the unobservable state and optimal control of the filtered state. Both finite and infinite time horizon problems are investigated.

Dynamic optimization over infinite-time horizon: Web-building strategy in an orb-weaving spider as a case study

Dynamic state-dependent models have been widely developed since 1990s for solving questions in evolutionary ecology. Up to now, these models were mainly run over finite-time horizon. However, for many biological questions an infinite-time horizon perspective could be more appropriate, especially when the end of the modeled period is state- rather than time-dependent. Despite this approach is widely used in the field of economics and operational research, thus far no work has been providing biologists with a general method to solve infinite-time horizon problems. Here we present such a method, through the exhaustive description of an algorithm that we implement to determine the strategy an organism should follow to reach a particular state as fast as possible while limiting mortality risk. To illustrate that method we explored web-building behavior in an orb-weaving spider. How are adult females predicted to build their successive webs to gain energy, grow, and lay their first clutch as fast as possible, without suffering from either predation or starvation? From this example, we first show how an optimal strategy over infinite-time horizon can be processed and selected. Second, we analyse variations of the optimal web-building strategy along with the spider's body weight and predation risk during web building. Our model yields two main predictions: (1) spiders reduce their web size as they are gaining weight due to body-mass-dependent cost of web-building behavior, and (2) this reduction in web size starts at lower weight under higher predation risk.

Mathematical Control Theory of Coupled PDEs

1R9. Mathematical Control Theory of Coupled PDEs. - I Lasiecka (Univ of Virginia, Charlottesville VA). SIAM, Philadelphia. 2002. 242 pp. Softcover. ISBN 0-89871-486-9. $60.00.Reviewed by GC Gaunaurd (Code AMSRL-SE-RU, Army Res Lab, 2800 Powder Mill Rd, Adelphi MD 20783-1197).The mathematics of control theory for a single Partial Differential Equation has been studied for some time. However, for more complex systems governed by systems of coupled PDEs, the available works are fewer. The tools developed for single PDE systems are usually inadequate for the analysis of coupled systems. New questions have been formulated, including how can one take advantage of the coupling in the model to improve the system performance? The propagation of some components of a system into some other, originate new phenomena via the coupling. The present book describes classes of coupled PDE models displaying the above-mentioned coupled properties and presents tools to analyze the resulting control problems. The ultimate goal of the book is said to be to provide a mathematical theory to guide the solution of three main problems: a) well-posedness and regularity, b) stabilization and stability, and c) optimal control and existence and uniqueness of some associated Riccati equations. The structural acoustics model is used toward the end of the book as the choice “example” to illustrate various coupling phenomena that appear in interconnected systems. There are wave equations in an acoustic medium that appear coupled to the plate or shell equations from which they are separated by an interface that is part of the boundary for the acoustic medium. The book has six chapters. It is only possible here to give their titles. The analysis starts with the well-posedness of 2nd-order nonlinear equations with boundary damping. It continues with a study of the stabilizability of nonlinear waves and plates. There is then a chapter on the uniform stability of structural acoustic models and another on Semi Group and PDE models for structural acoustic control problems. The final two chapters deal with feedback noise control for finite and infinite time-horizon problems. This results in detailed studies of certain pertinent Riccati equations. Mathematical Control Theory of Coupled PDEs has 242 pages, with no figures and no computed results. This is a pure mathematical treatment full of theorems, lemmas, assumptions, propositions, and countless corollaries. It is the author’s stated hope that it will be of use to applied acousticians and theoretical engineers, but this is an unrealistic expectation. The connection to structural acoustics is buried in a sea of theorems with little applicability. There are over 200 references, but about half are by the author herself and one of her associates. The couple of works mentioned which are authored by well-known acousticians, such as C Fuller and the textbook by P Morse and U Ingard, are the only ones that come from the regular acoustic literature. Therefore, this mathematical document will be mostly of interest to other mathematicians carrying on research on these obscure topics. This is certainly not a textbook, but perhaps could be a reference mathematics book for some institutional libraries. With the current emphasis on relevance required by the Army Research Office (ARO), now a part of the Army Research Lab, this reviewer was surprised to learn that they sponsored this work.

Solution of perfect foresight saddlepoint problems: a simple method and applications

The paper proposes a method to compute adjustment dynamics that occur in infinite time horizon problems of deterministic optimal control. Without loss of accuracy, this method is easier to implement than existing ones. Discussed applications are the standard neoclassical growth model and the general two-sector growth model with cyclical adjustment on a two-dimensional manifold.

On the Continuous Dependence with Respect to Sampling of the Linear Quadratic Regulator Problem for Distributed Parameter Systems

The convergence of solutions to the discrete- or sampled-time linear quadratic regulator problem and associated Riccati equation for infinite-dimensional systems to the solutions to the corresponding continuous time problem and equation, as the length of the sampling interval (the sampling rate) tends toward zero (infinity) is established. Both the finite- and infinite-time horizon problems are studied. In the finite-time horizon case, strong continuity of the operators that define the control system and performance index, together with a stability and consistency condition on the sampling scheme are required. For the infinite-time horizon problem, in addition, the sampled systems must be stabilizable and detectable, uniformly with respect to the sampling rate. Classes of systems for which this condition can be verified are discussed. Results of numerical studies involving the control of a heat/diffusion equation, a hereditary or delay system, and a flexible beam are presented and discussed.

Optimal quadratic control of jump linear systems with separately controlled transition probabilities

Abstract The optimal control of discrete-time jump systems that would be linear but for randomly jumping parameters are considered. These parameter jumps, which are indexed by a finite state form process, can be used to model systems subject to component failures or structural changes. The control of such systems when the jump probabilities can be controlled by choice of a finite-valued input is considered. This input (form control) can be thought of as corresponding to a maintenance or repair policy. The optimal control problem with quadratic costs (whose weighting matrices depend on the jumping parameters and form control) is formulated for such systems. The optimal solution for finite and infinite time horizon problems is developed. In addition, sufficient conditions for the existence of a steady-state, stabilizing solution to the infinite time problem are given in terms of appropriately defined controllability and observability properties.

Optimal quadratic control of jump linear systems with separately controlled transition probabilities

Abstract The optimal control of discrete-time jump systems that would be linear but for randomly jumping parameters are considered. These parameter jumps, which are indexed by a finite state form process, can be used to model systems subject to component failures or structural changes. The control of such systems when the jump probabilities can be controlled by choice of a finite-valued input is considered. This input (form control) can be thought of as corresponding to a maintenance or repair policy. The optimal control problem with quadratic costs (whose weighting matrices depend on the jumping parameters and form control) is formulated for such systems. The optimal solution for finite and infinite time horizon problems is developed. In addition, sufficient conditions for the existence of a steady-state, stabilizing solution to the infinite time problem are given in terms of appropriately defined controllability and observability properties.

Optimal feedback control of bilinear systems

Feedback synthesis of optimal constrained controls for single-input bilinear systems is considered. Quadratic cost functionals (with and without quadratic control penalization) are modified by the inclusion of additional nonnegative state penalizing functions in the respective cost integrands. The latter functions are chosen so as to regularize the problems, in the sense that feedback solutions of particularly simple form are obtained. Finite and infinite time horizon problem formulations are treated, and associated aspects of feedback stabilization of bilinear systems are discussed.

On income fluctuations and capital gains

We consider the infinite time horizon problem of asymptotically maximizing the expected accumulated discounted utility in a one-good production economy. The available capital in a given period is given by the production of the previous period plus a random variable. The product of the discount and interest factors is either (1) greater than or (2) equal to one. Under (1) the optimal policy exists under certain conditions and always under (2). The optimal capital sequence almost surely goes to infinity. Under (1) with conditions on the utility one almost surely reaches a capital level above which the sequence is increasing.

Optimal Saturating Control of Distributed Parameter Systems

Optimization of linear distributed parameter systems, with finite dimensional control input subject to component saturation constraints, is considered. An integral cost is introduced, with integrand comprised of quadratic and non-quadratic state penalty terms. The latter (non-quadratic) terms are chosen so as to regularize the problem in the sense that an optimal solution, of a simple feedback form, is obtained. Finite and infinite time horizon problem formulations are examined and two examples are presented.

Some results on “an income fluctuation problem”

A consumer at each period, given the income available, y, has to decide how much to consume and save. If he consumes c ⩾ 0 units he gets u( c) units of satisfaction or utility, and if x = y − c ⩾ 0 is the amount saved then the available income in the next period is rx + ω k, where ω k is a random variable, and r is an interest factor that is assumed to be known with certainty. Infinite time horizon problems are considered, and it is shown that if 0 < δr < 1, where 0 < δ < 1 is a discount factor, then the limiting policy is optimal. Questions about the behavior of the stock level, such as boundness, are considered, and an example is given that shows that the stock level might converge almost surely to infinity. Finally an economic explanation is given.