Optimizing steady state Markov chains by state reduction

Abstract

Many service systems have a parameter which can be changed continuously within a certain range. In queueing, for instance, one may be able to change the service rate by means of faster servers, or the arrival rate through advertising. The question to be addressed is how to choose the value of the parameter in order to maximize rewards. In this paper, service systems are formulated as Markov chains in equilibrium. The optimum is then found by Newton's method. This requires one to determine the first and second derivative of the rewards, and an effective method for doing this is proposed. The method is based on state reduction, which is a technique for finding equilibrium probabilities for finite and even for infinite state Markov chains. The optimization technique is then used to obtain a number of curves involving two simple service systems.