Long-run Average Criterion Research Articles

Empty container management in a port with long-run average criterion

The empty container allocation problem in a port is related to one of the major logistics issues faced by distribution and transportation companies: the management of importing empty containers in anticipation of future shortage of empty containers or exporting empty containers in response to reduce the redundance of empty containers in this port. We considered the problem to be a nonstandard inventory problem with positive and negative demands at the same time under a general holding-penalty cost function and one-time period delay availability for full containers just arriving at the port. The main result is that there exists an optimal pair of critical policies for the discounted infinite-horizon problem via a finite-horizon problem, say ( U, D). That is, importing empty containers up to U when the number of empty containers in the port is less than U, or exporting the empty containers down to D when the number of empty containers is more than D, doing nothing otherwise. Moreover, we obtain the similar result over the average infinite horizon.

Ergodic control of reflected diffusions with jumps

We discuss ergodicity properties of a controlled jumps diffusion process reflected from the boundary of a bounded domain. The control parameters act on the drift term and on a first-order-type jump density. The controlled process is generated via a Girsanov change of probability, and a long-run average criterion is optimized. An optimal stationary feedback is constructed by means of the Hamilton-Jacobi-Bellman equation.

A martingale approach to the slow server problem

A Markov queueing system having heterogeneous servers under a long-run average criterion is analyzed. A direct proof of the optimality of a stationary, Markov policy is given using martingale methods. Simultaneously, the problem is reduced to a linear programming problem. Analysis of the LP for a system having finite queueing length shows the optimal policy is not always of threshold type.

A martingale approach to the slow server problem

A Markov queueing system having heterogeneous servers under a long-run average criterion is analyzed. A direct proof of the optimality of a stationary, Markov policy is given using martingale methods. Simultaneously, the problem is reduced to a linear programming problem. Analysis of the LP for a system having finite queueing length shows the optimal policy is not always of threshold type.

Self-tuning control of diffusions without the identifiability condition

The self-tuning method of adaptive control for diffusions consists of estimating the unknown parameter on line and using its current estimate as the true parameter for the selection of the control at each time. The a.s. optimality of this scheme for the ergodic or long-run average criterion can be established under an identifiability condition on the system, but may fail otherwise. We present a modified self-tuning scheme along the lines of the Kumar-Becker-Lin scheme for Markov chains and prove its a.s. optimality. Several heuristic issues related to this scheme are also discussed.

More on the “anti-folk theorem”

For discounted repeated games with unobservable individual deviations, Kaneko's “anti-folk theorem” states that the set of Nash-equilibrium plays coincides with the set of sequences of one-shot Nash-equilibrium plays. When the payoff criterion is long-run average, however, Kaneko's characterization is of a different sort. Here we show that with some additional topological assumptions a version of the anti-folk theorem is available under the long-run average criterion which is parallel to the characterization under the discounting criterion.

Two competing queues with linear costs and geometric service requirements: the μc-rule is often optimal

A discrete-time model is presented for a system of two queues competing for the service attention of a single server with infinite buffer capacity. The service requirements are geometrically distributed and independent from customer to customer as well as from the arrivals. The allocation of service attention is governed by feedback policies which are based on past decisions and buffer content histories. The cost of operation per unit time is a linear function of the queue sizes. Under the model assumptions, a fixed prioritization scheme, known as the μc-rule, is shown to be optimal for the expected long-run average criterion and for the expected discounted criterion, over both finite and infinite horizons. Two different approaches are proposed for solving these problems. One is based on the dynamic programming methodology for Markov decision processes, and assumes the arrivals to be i.i.d. The other is valid under no additional assumption on the arrival stream and uses direct comparison arguments. In both cases, the sample path properties of the adopted state-space model are exploited.

Two competing queues with linear costs and geometric service requirements: the μc-rule is often optimal

A discrete-time model is presented for a system of two queues competing for the service attention of a single server with infinite buffer capacity. The service requirements are geometrically distributed and independent from customer to customer as well as from the arrivals. The allocation of service attention is governed by feedback policies which are based on past decisions and buffer content histories. The cost of operation per unit time is a linear function of the queue sizes. Under the model assumptions, a fixed prioritization scheme, known as the μc-rule, is shown to be optimal for the expected long-run average criterion and for the expected discounted criterion, over both finite and infinite horizons. Two different approaches are proposed for solving these problems. One is based on the dynamic programming methodology for Markov decision processes, and assumes the arrivals to be i.i.d. The other is valid under no additional assumption on the arrival stream and uses direct comparison arguments. In both cases, the sample path properties of the adopted state-space model are exploited.

Two competing queues with linear costs: the μc-rule is often optimal

A state-space model is presented for a queueing system where two classes of customer compete in discrete-time for the service attention of a single server with infinite buffer capacity. The arrivals are modelled by an independent identically distributed random sequence of a general type while the service completions are generated by independent Bernoulli streams; the allocation of service attention is governed by feedback policies which are based on past decisions and buffer content histories. The cost of operation per unit time is a linear function of the queue sizes. Under the model assumptions, a fixed prioritization scheme, known as the μc -rule, is shown to be optimal when the expected long-run average criterion and the expected discounted criterion, over both finite and infinite horizons, are used. This static prioritization of the two classes of customers is done solely on the basis of service and cost parameters. The analysis is based on the dynamic programming methodology for Markov decision processes and takes advantage of the sample-path properties of the adopted state-space model.

Semi-Markov decision processes with polynomial reward

A semi-Markov decision process, with a denumerable multidimensional state space, is considered. At any given state only a finite number of actions can be taken to control the process. The immediate reward earned in one transition period is merely assumed to be bounded by a polynomial and a bound is imposed on a weighted moment of the next state reached in one transition. It is shown that under an ergodicity assumption there is a stationary optimal policy for the long-run average reward criterion. A queueing network scheduling problem, for which previous criteria are inapplicable, is given as an application.