State Substitution and the Estimation of Transition Probabilities

Abstract

Variants of Markovian decision models are many. Of those concerned with the estimation of transition probabilities, few deal with the case of limited information in which only data on market shares are available. Where such investigations have been undertaken, the assumption of constant transition probabilities has been insurmountable. This paper offers a simple heuristic technique to estimate nonstationary transition probabilities, especially in the case where fluctuations in market shares are felt to reflect the effects of secondary considerations external to the transition probabilities as well as effects of the probabilities themselves. Dynamic and quadratic programming are the suggested tools of analysis.