The control of a finite dam with penalty cost function: Markov input rate

Abstract

The long-run average cost per unit time of operating a finite dam controlled by a policy (Lam Yeh (1985)) is determined when the cumulative input process is the integral of a Markov chain. A penalty cost which accrues continuously at a rate g(X(t)), where g is a bounded measurable function of the content, is also introduced. An example where the input rate is a two-state Markov chain is considered in detail to illustrate the computations.