Long-run Average Cost Criterion Research Articles

Control of Markov Chains with Long-Run Average Cost Criterion: The Dynamic Programming Equations

The long-run average cost control problem for discrete time Markov chains on a countable state space is studied in a very general framework. Necessary and sufficient conditions for optimality in terms of the dynamic programming equations are given when an optimal stable stationary strategy is known to exist (e.g., for the situations studied in [Stochastic Differential Systems, Stochastic Control Theory and Applications, IMA Vol. Math. App. 10, Springer-Verlag, New York, Berlin, 1988, pp. 57–77]). A characterization of the desired solution of the dynamic programming equations is given in a special case. Also included is a novel convex analytic argument for deducing the existence of an optimal stable stationary.strategy when that of a randomized one is known.

A note on the optimal control of a cash balance problem

This paper provides an analytical solution to a cash management problem when cash income and demand are described by Compound Poisson processes. The paper generalizes past results in the cash management literature to arbitrary income and demand distribution functions. Further, our results can be applied as well in the area of banking. Throughout the paper we restrict attention to the family of control barrier policies. These consist in hedging cash up to a critical level and investing all incoming cash exceeding this level. We employ a long-run average cost criterion to determine an optimal control barrier. A diffusion approximation of the cash level process (income less demand) is used to obtain a simpler expression for the average cost and to yield a closed form solution to the optimal control barrier. For demonstration purposes, an example is resolved.