Average Cost Criterion Research Articles

Computational procedures for stochastic multi-echelon production systems

This paper is concerned with the numerical evaluation of multi-echelon production systems. Each stage requires a fixed predetermined leadtime; furthermore, we assume a stochastic, stationary end-time demand process. In a previous paper, we have developed an analytical framework for determining optimal control policies for such systems under an average cost criterion. The current paper is based on this analytical theory but discusses computational aspects, in particular for serial and assembly systems. A hierarchical (exact) decomposition of these systems can be obtained by considering echelon stocks and by transforming penalty and holding costs accordingly. The one-dimensional problems arising after this decomposition however involve incomplete convolutions of distribution functions, which are only recursively defined. We develop numerical procedures for analysing these incomplete convolutions; these procedures are based on approximations of distribution functions by mixtures of Erlang distributions. Combining the analytically obtained (exact) decomposition results with these numerical procedures enables us to quickly determine optimal order-up-to levels for all stages. Moreover, expressions for the customer service level of such a multi-stage are obtained, yielding the possibility to determine policies which minimize average inventory holding costs, given a service level constraint.

Average cost Markov Decision Processes: Optimality conditions

We are concerned in this paper with discrete-time Markov Decision Processes (MDPs) with Borel state and action spaces X and A, respectively, and the long run expected average cost criterion. When X is a denumerable set, man y necessary and/or sufficient conditions for the existence of optimal control policies are known. However, when X is a Borel space (i.e., a Borel subset of a complete separable metric space), most of the available results impose on the MDP very restrictive topological conditions (e.g., compactness) and/or strong recurrence assumptions (such as Doeblin's condition); see, e.g., [4,9, 12] and their references. Another related work is [7] where we have studied MDPs from the viewpoint of the recurrence (or ergodicity) properties of the state process. ln the present paper, however, we are concerned with the existence of average optimal policies by looking at (static) optimization problems (see condition CS in Section 4) related-in sorne cases equivalent-to the existence of a bounded solution to the so-called Optimality Equation (see C4 in Section 4). These optimization problems are dual in the sense that, under appropriate conditions, the existence of an optimal solution to one of the problems implies existence of an optimal

Optimal Manufacturing of a Two Product Mix

We consider a machine that works constantly and produces two types of fluid products, one type of product at a time. Whenever there is a positive amount of both types of products, the products are mixed immediately and shipped away. Whenever mixing is infeasible the product that is currently being produced accumulates inventory. The goal is to minimize the long-run average as well as the discounted costs of running such an operation. Inventory incurs a linear holding cost for each type of product. Also, whenever switching from the production of one type of product to the other, a fixed switching cost is incurred. It is found that a stationary periodic policy is optimal. The optimal solution for the long-run average cost criterion is described by closed form formulas. As a special case, we show that the (s, S) policy for the textbook EOQ inventory model is indeed optimal over the class of all policies. Also, a theorem that proves the optimality of cyclic policies within a particular set of policies is introduced in a very general setting.

A counterexample on the optimality equation in Markov decision chains with the average cost criterion

We consider Markov decision processes with denumerable state space and finite control sets; the performance index of a control policy is a long-run expected average cost criterion and the cost function is bounded below. For these models, the existence of average optimal stationary policies was recently established in [11] under very general assumptions. Such a result was obtained via an optimality inequality. Here, we use a simple example to prove that the conditions in [11] do not imply the existence of a solution to the average cost optimality equation.

On the optimality of (s, S)-strategies in a minimax inventory model with average cost criterion

A stochastic dynamic inventory problem is considered in which the distribution function of the demand and the holding and shortage cost depend on an unknown parameter. This problem is interpreted as a Markov game with complete information, It is shown that for the average cost criterion as for the discounted cost criterion an optimal (s, S)-strategy exists. Bounds for the parameters of such a strategy are given.

Remarks on the existence of solutions to the average cost optimality equation in Markov decision processes

Necessary conditions are given for the existence of a bounded solution to the optimality equation arising in Markov decision processes, under a long-run, expected average cost criterion. The relationships of some of our results to known sufficient conditions are also shown.

An analytical theory of multi‐echelon production/distribution systems

In this paper, we study inventory control problems arising in multi‐echelon production/distribution chains. In these chains, material is delivered by outside suppliers, proceeds through a number of manufacturing stages, and is distributed finally among a number of local warehouses in order to meet market demand. Each stage requires a fixed leadtime; furthermore, we assume a stochastic, stationary end‐item demand process.The problem to balance inventory levels and service degrees can be modelled and analyzed by defining appropriate cost functions. Under an average cost criterion, we study the three most important structures arising in multi‐echelon systems: assembly systems, serial systems and distribution systems. For all three systems, it is possible to prove exact decomposition results which reduce complex multi‐dimensional control problems to simple one‐dimensional problems. In addition, we establish the optimality of base‐stock control policies.

Analysis and computation of ( n, N)-strategies for maintenance of a two-component system

In this paper ( n, N)-strategies are investigated for the maintenance of a two component series system. Vergin and Scriabin [8] provided some numerical evidence on the near-optimality of this type of policies. In a recent paper, Özekici [5] gave a characterization of the structure of the optimal policy and its possible deviations from the ( n, N)-structure for the discounted cost criterion. The same kind of structure is shown to hold for the average cost criterion. In view of the complicated form of the optimal policy and the near-optimality of the ( n, N)-policies we present a fast computational method to compute the average costs under a given ( n, N)-policy. This method is based on a well-known embedding technique. Moreover, a heuristic based on this computational method, is presented by which the optimal values of n and N can be determined.

On the compactness method in general ergodic impulsive control of markov processes

In this paper the impulsive control of Feller Markov processes on compact state space with long run average cost criterion is studied. Under the assumption of compactness of the resolvent operator the optimal strategies corresponding to general cost for impulses are constructed. Also the case of purely jump Markov processes is considered

Optimal Control of the Vacation Scheme in an M/G/1 Queue

In this article the M/G/1 queue with server vacations is considered with the assumption that the decision whether or not to take a new vacation, when the system is empty, depends on the number of vacations already taken through a random outcome. Both descriptive and optimization issues are considered, where the latter is done under the expected long-run average cost criterion with linear holding costs, fixed setup costs and a concave piecewise linear reward function for being on vacation. The optimization problem results in an infinite dimensional fractional program of which the solution yields a (deterministic) policy of the control limit type.

Analysis of an identification algorithm arising in the adaptive estimation of Markov chains

We investigate an algorithm applied to the adaptive estimation of partially observed finite-state Markov chains. The algorithm utilizes the recursive equation characterizing the conditional distribution of the state of the Markov chain, given the past observations. We show that the process “driving” the algorithm has a unique invariant measure for each fixed value of the parameter, and following the ordinary differential equation method for stochastic approximations, establish almost sure convergence of the parameter estimates to the solutions of an associated differential equation. The performance of the adaptive estimation scheme is analyzed by examining the induced controlled Markov process with respect to a long-run average cost criterion.

Ergodic control of multidimensional diffusions, II: Adaptive control

The self-tuning scheme for the adaptive control of a diffusion process is studied with long-run average cost criterion and maximum likelihood estimation of parameters. Asymptotic optimality under a suitable identifiability condition is established under two alternative sets of hypotheses—a Lyapunov-type stability criterion and a condition on cost which penalizes instability.

Shepp statistic for Markov chains application to a long-run average cost criterion

This paper is a generalization to Markov chains of the work of Shepp [6] in the i.i.d case. Shepp studies the limiting values of the averagesTn= (Sn+f(n)–Sn)/f(n) whereSn=X0+X1+ · ·· +Xn,X0= 0,n= 1, 2, ···, is a sum of mutually independent and identically distributed random variables. The functionftakes positive integer values and non-decreasingly tends to infinity. Here we take a class of functionsfin central positionf(n) = [clogn],c> 0,n= 1, 2, ···. There are many refinements of the functionfin the i.i.d case [1], [2]. Here we consider the more general case whereX1,· ··,Xnis an irreducible and recurrent Markov chain. The state space of the chain is either compact or countable.

Shepp statistic for Markov chains application to a long-run average cost criterion

This paper is a generalization to Markov chains of the work of Shepp [6] in the i.i.d case. Shepp studies the limiting values of the averages Tn = (Sn + f(n) – Sn )/f(n) where Sn = X0 + X 1+ · ·· + Xn , X 0 = 0, n = 1, 2, ···, is a sum of mutually independent and identically distributed random variables. The function f takes positive integer values and non-decreasingly tends to infinity. Here we take a class of functions f in central position f(n) = [c log n], c > 0, n = 1, 2, ···. There are many refinements of the function f in the i.i.d case [1], [2]. Here we consider the more general case where X 1 , · ··, Xn is an irreducible and recurrent Markov chain. The state space of the chain is either compact or countable.

Control of Markov Chains with Long-Run Average Cost Criterion: The Dynamic Programming Equations

The long-run average cost control problem for discrete time Markov chains on a countable state space is studied in a very general framework. Necessary and sufficient conditions for optimality in terms of the dynamic programming equations are given when an optimal stable stationary strategy is known to exist (e.g., for the situations studied in [Stochastic Differential Systems, Stochastic Control Theory and Applications, IMA Vol. Math. App. 10, Springer-Verlag, New York, Berlin, 1988, pp. 57–77]). A characterization of the desired solution of the dynamic programming equations is given in a special case. Also included is a novel convex analytic argument for deducing the existence of an optimal stable stationary.strategy when that of a randomized one is known.

The threshold policy in theM/G/1 queue with server vacations

This article deals with the M/G/1 queue with server vacations in which the return of the server to service depends on the number of customers present in the system. The main goal is optimization, which is done under the average cost criterion in the multiple- and single-vacation models as well as for the “total cost for one busy cycle” criterion in the multiple-vacation case. Expressions that characterize the optimal number of customers, below which the server should not start a new service period, are exhibited for the various cases. It is found that under the average cost criterion, the expression may be universal in the sense that it may hold for a general class of problems including such that arise in production planning and inventory theory (for the particular cost structure discussed).

Quadratic control for linear periodic systems

We propose a new quadratic control problem for linear periodic systems which can be finite or infinite dimensional. We consider both deterministic and stochastic cases. It is a generalization of average cost criterion, which is usually considered for time-invariant systems. We give sufficient conditions for the existence of periodic solutions.

Optimal adaptive inventory control for a multi-location model with redistribution

In Kochel 1982 a multi-location inventory model with redistribution is considered for the discounted cost criterion with full information on the demand characteristics. In the present paper the case that the distribution function of the demand depends on an unkown parameter is investigated. For this results on the discounted cost criterion are carried over to the average cost criterion. The so-called identification scheme from adaptive control theory for discrete-time Markov processes is applied and average optimality for the resulting adaptive inventory policy is shown. Simulation results show the practicability of that approach.

A variance minimization problem for a Markov decision process

In the steady state of a discrete time Markov decision process, we consider the problem to find an optimal randomized policy that minimizes the variance of the reward in a transition among the policies which give the mean not less than a specified value. The problem is solved by introducing a parametric Markov decision process with average cost criterion. It is shown that there exists an optimal policy which is a mixture of at most two pure policies. As an application, the toymaker's problem is discussed.

Elimination of eanlomization in semi-Markov decision models with average cost criterion

Average cost optimization problem for a semi-Markov decision model with a countable state space and an arbitrary decision space is considered. The Conditions without optimality equation are discussed when the next proposition is true:If there exists an optimal stationary strategythen there exists a nonrandomized oneThe results obtained here and in [7] allow to formulate sufficient conditions, convenient in queueing theory, for optimality of a stationary nonrandomized strategy.