Performance gradient estimation for the very large finite Markov chains

Abstract

Using an embedded Markov chain, the steady-state performance gradient estimation for very large Markov chains is decomposed into two smaller problems, one solved by the stochastic recursive (SR) method and the other by the likelihood ratio (LR) method. The combination, named SR-LR, can have several magnitudes of lower space requirement than the SR and several magnitudes of lower time consumptions than the LR. Both the LR and the SR are the extreme cases of the combined algorithm. The authors demonstrate the SR-LR methods on Markov chains with hundreds of millions of states, which are created using queuing networks. >